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PREFACE 


The chief purpose of this volume is to present a general treatment of 
factor analysis including an objective basis for comparing various forms of 
solution. To accomplish this a thorough examination has been made of the 
logical and mathematical foundations underlying the methods of analysis 
employed. The treatment throughout has been made as rigorous as pos- 
sible with the hope that factor analysis would emerge as a sound branch of 
statistical theory rather than as a series of disconnected methods of appli- 
cation. A considerable amount of mathematics has been found necessary 
for such rigorous treatment. The text, however, has been made self-con- 
tained in large measure by the treatment of such subjects as higher dimen- 
sional geometry and the fundamentals of matrix theory. 

Another aim of the text is the practical one of supplying the methods of 
analysis in a very simple, readable form for the worker who is not concerned 
with the rigorous foundation. Such an analyst would need only a general 
knowledge of the various forms of solution, as described in the text, and he 
could then follow the detailed outlines for computation, which are given in 
the appropriate appendixes. Complete checks on all the arithmetical proc- 
esses are also given throughout these appendixes. 

In Part I the analytic and geometric bases for factor analysis are set 
forth. Certain fundamental statistics are introduced in Chapter II, the most 
important of which include the assumed factorial composition of variables 
and the distinction between patterns and structures. Inasmuch as geometry 
is a convenient form not only for presenting the basic theorems of factor 
analysis but also for clarifying many ideas in the actual solutions, Chap- 
ter III is devoted entirely to that subject. The fundamental theorems basic 
to the various methods are given in Chapters III and IV. A new approach 
to the determination of the common-factor space is presented in Chapter IV. 

In order to furnish an objective basis for selecting a given form of solu- 
tion, a set of statistical criteria are enumerated in Chapter V. When cer- 
tain of these standards are postulated, several “preferred” solutions are 
disclosed. These are presented formally in Chapter V, and the distinguish- 
ing characteristics of each type are indicated in detail. The practical ana- 
lyst who does not desire to read the complete development of each method 
should become thoroughly acquainted with the properties of the types of 
solution treated in this chapter (and also the oblique form of solution of 
Chap. XI) so that he can make an intelligent choice of a particular form. 
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Chapter V is of fundamental importance for a clear understanding of the 
problem of choice of a particular type of factor solution, and the synthetic 
treatment of this chapter should add unity to the subject of factor analysis. 

The complete theoretical development of the preferred types of solution 
is given in Parts II and III. The theory underlying each form is presented, 
and this is followed by illustrative examples. Among the new methods in- 
troduced in the present text are the averoid modification of the centroid 
method in Chapter VIII and the objective procedure for oblique solutions 
of Chapter XI. Sampling formulas of factor coefficients and for residuals 
have also been developed in Chapter VI. Complete forms of analysis, in- 
cluding detailed steps of procedure for each method, have been given in 
Appendixes C, D, and E. These should be of chief value to the nonmathe- 
matical student who may find portions of the theoretical treatment some- 
what difficult. 

The evaluation of factors and relationships among factors are dealt with 
in Part IV. A synthetic treatment of several methods for estimating factors 
from the observed variables is given in Chapter XII. The relative advan- 
tages and disadvantages of each method are indicated, so that the most de- 
sirable procedure can be readily selected. A routine scheme for the estima- 
tion of factors is given in Appendix G. In the final ch 
between two factor solutions obtained by different methods of analysis are 
discussed. Then, to assist the practical analyst in employing the various 
methods described throughout the book, a general outline of procedure is 
suggested in 13.4. _ 
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factor analysis. The instructor who uses this volume as a text will probably 
want to exercise his judgment as to the portions to be omitted. It is sug- 
gested, however, that all basic theorems be considered and illustrated even 
though the proofs are omitted. 

No attempt has been made to give a historical account of the develop- 
ment of factor analysis. This has been omitted partly because of the greatly 
added bulk and partly because such an aecount would involve both statisti- 
cal method and psychological theories and thus would be outside the scope 
of the present general statistical treatment. For similar reasons it has also 
been found necessary to omit the treatment of a number of alternative ap- 
proaches to factor analysis such as the "Sampling Theory of Intelligence," 
“Cluster Analysis," and ‘Inverted Factor Analysis.” 

A number of practical illustrations of all the methods are presented 
throughout the text. Complete calculations are exhibited in order to clarify 
the various procedures. No additional exercises have been included, but it 
is planned to supplement the present volume with an exercise manual. This 
will include simple exercises, practical applications, and questions designed 
to increase the usefulness of the present volume as a textbook. For pur- 
poses of reference a bibliography of contributions to the methods of factor 
analysis is also presented. This includes not only works on factorial meth- 
ods and some of their major applications but also related mathematical 
treatises. 

Each section in the text is distinguished by a number in bold-face type 
consisting of the number of the chapter in which the section occurs, pre- 
fixed to the number of the section in that chapter and separated from it by 
a period; e.g., 2.10 indicates the tenth section of the second chapter. Simi- 
larly tables, figures, and equations are distinguished first with the number 
of the chapter in which they occur and then, separated by a period, with 
their serial number within the chapter; e.g., Table 7.3 refers to the third 
table of Chapter VII and “equation (6.2)” refers to the second equation of 
Chapter VI. A few page references are also included, but the above notation 
will be found convenient for rapid cross-reference. 

The authors are greatly indebted to the Social Science Research Commit- 
tee of the University of Chicago for grants which made possible the research 
required for the present volume, They are especially indebted to Professor 
Louis Wirth for his suggestion that the present synthetic treatment be un- 
dertaken and for his advice and encouragement during the lengthy prepara- 
tion of the manuscript. To the penetrating criticisms of Dr. A. L. O’Toole 
are due many improvements in the manuscript. The writers are also in- 
debted to Dr. L. R. Wilcox for criticism of Chapter III on geometry and for 
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his suggestion of the set-theory notations. To Dr. Lee J. Cronbach, who 
read several chapters of the original manuscript, they are grateful for helpful 
suggestions and criticisms. 

Karu J. HorziNGER 


Harry H. HARMAN 
UNIVERSITY OF CHICAGO 


May 1941 
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PART I 


ANALYTIC AND GEOMETRIC BASES 
FOR FACTOR ANALYSIS 


CHAPTER I 
INTRODUCTION 


1.1. Nature of Factor Analysis 


Factor analysis is a branch of statistical theory concerned with the reso- 
lution of a set of descriptive variables in terms of a small number of cate- 
gories or factors. This resolution is accomplished by the analysis of the 
intercorrelations of the variables. A satisfactory solution will yield factors 
which convey all the essential information of the original set of variables. 
The chief aim is thus to attain scientific parsimony or economy of de- 
scription. 

This aim should not be construed to mean that factor analysis necessarily 
attempts to discover the “fundamental” or "basic" categories in a given 
field of investigation such as psychology. It would be very desirable, of 
course, to base such an analysis upon a set of variables which measures all 
possible mental aspects of a given population as completely and accurately 
as possible. Even in such a case, however, the factors would not be com- 
pletely fundamental because of the omission of important measures which 
were not yet devised. While the goal of complete description cannot be 
reached theoretically, it may be approached practically in a limited field of 
investigation where a relatively small number of variables is considered 
exhaustive. In all cases, however, factor analysis does give a simple inter- 
pretation of a given body of data and thus affords a fundamental descrip- 
tion of the particular set of variables analyzed. 

The essential purpose of factor analysis has been well expressed by Pro- 
fessor Truman L. Kelley: ‘‘There is no search for timeless, spaceless, popu- 
lationless truth in factor analysis; rather, it represents a simple, straight- 
forward problem of description in several dimensions of a definite group 
functioning in definite manners, and he who assumes to read more remote 
verities into the factorial outcome is certainly doomed to disappointment." * 


1.2. Applications and Problems 


The applications of factor analysis up to the present time have been 
chiefly in the field of psychology, because the methods were invented by 
psychologists for dealing with certain of their problems. A factor analysis 

* "Comment on Wilson and Worcester's *Note on Factor Analysis,’ ” Psychometrika, 
1940, p. 120. 
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may lead to some theory suggested by the form of the solution, and con- 
versely one may formulate a theory and verify it by an appropriate form 
of factorial solution. The latter approach is illustrated by Professor Charles 
Spearman's theory that “all branches of intellectual activity have in common 
one fundamental function (or group of functions) whereas the remaining or 
specific elements of the activity seem in every case to be wholly different from 
that in all others."* He showed that if certain relationships (the tetrads 
defined in 4.3) exist among the correlations, all the variables can be re- 
solved into linear expressions involving only one general factor and an 
additional factor unique to each variable. These relationships furnish the 
statistical verification of the “Two-Factor Theory.” If a set of psychologi- 
cal variables yields correlation coefficients which do not satisfy the preced- 
ing relationships, then a more complex theory may be postulated. This may 
require several common factors in the statistical description of the vari- 
ables. 

Recently, Kelley has described a method involving factor analysis which 
may well be used in the selection and classification of army personnel.t Of 
fundamental importance for this analysis is the selection of variables which 
have definite social significance. Two types of variables are involved: first, 
those which are measures of social utility and, second, those which are con- 
cerned with measuring the well-being of the individual. The former meas- 
ures are selected because of their bearing on vocational success, while the 
latter are chosen to reflect individual happiness. From such a set of vari- 
ables several factors are obtained. Then the profile of an individual can be 
compared with that of the average participant in the type of job being con- 
sidered. Kelley’s primary purpose was not “to suggest the measures to be 
employed, nor even the detailed principles for their selection, but to give a 
method for utilizing them to their fullest . - - - in the service of a State dedi- 
cated to the preservation of human liberties and rights." 

Psychologists are planning experiments employing the techniques of fac- 
tor analysis to determine a small number of 
the human mind as completely as possible. 
torial analysis of a large battery of tests in 
factors. Then the tests which best measu 
revised tests based upon these, may be se 
"factors of mind." However, only to th 


* "General Intelligence, Objectively Determined and Measured,” 


Amer. J. 

1904, p. 201. mer. J. Psychol., 
T Talents and Tasks: Their Conjunction in a Democracy for Wholes - 

tional Defense, 1940. ome Living and Na- 


1 Ibid., p. 48. 
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agree that the tests selected are the “right tests" can they be said to be actu- 
al measures of the factors. Such "factor tests" should be of a “pure” nature, 
differing widely from one another so as to cover the entire range of mental 
activity. To the present time, several major studies* have been undertaken 
to identify factors from large sets of tests. The tests which best measure 
each of the factors, however, are still far from being "pure." A great deal 
of research in test construction is still required before tests can be found 
which can be regarded as even approximating the “factors of the mind.” 

The methods of factor analysis have been successfully applied, in recent 
years, not only in psychology but also in such varied fields as political sci- 
ence, business, and medicine. Sometimes, in the study of a particular prob- 
lem, variables from several different fields may be employed. Thus, in an 
attempt to discover the underlying causes for poor reading, psychological, 
physical, and medical evidence has been pooled for factorial analysis. In a 
widely different field a physicist has turned to factor analysis to discover 
some simple explanation of the intensity of cosmic rays under varying con- 
ditions of temperature and air pressure at different altitudes. A factor anal- 
ysis of stock prices of railroad equipment, automotive parts, and oils has 
been made to arrive at the fundamental indices of this type of business ac- 
tivity. It has also been proposed to employ factor analysis in medicine in 
order to obtain a simple functional classification of substances producing 
allergy. These are only a few of the types of practical applications that may 
be expected as the methods of factor analysis become more widely known. 

The foregoing applications of factor analysis are concerned mainly with 
testing hypotheses in scientific investigations. Another use, considered by 
some to be the chief value of factor analysis, is to supplement conventional 
statistical techniques. By means of factor analysis the computation of mul- 
tiple correlation coefficients, partial correlation coefficients, and regression 
coefficients can be greatly simplified when many variables are involved. In 
the problem of studying the relationship between two sets of variables factor 
analysis can again be employed. The best linear function of the variables in 
each set is obtained by factorial methods, and then the correlation between 
these composites gives what is known as the canonical correlation.t Such a 
correlation is the maximum possible between the two sets of variables. This 
technique should be of great value in test construction, inasmuch as it fur- 
nishes the investigator with improved methods for obtaining more valid 
and reliable pools of subtests. 


* L. L. Thurstone, Primary Mental Abilities, 1938; K. J. Holzinger, Preliminary Re- 
Ports on Spearman-Holzinger Unitary Trait Study, Nos. 1-9, 1934-36. 


t Harold Hotelling, “Relations between Two Sets of Variates," Biometrika, 1935. 
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The present text is concerned primarily with the exposition of various 
procedures in factor analysis. These methods are completely illustrated by 
numerical examples taken from psychology, biometry, and political science. 
These examples are used chiefly to clarify the theoretical treatment rather 
than to exhibit the practical usefulness of factor analysis in the broader type 
of application mentioned above. 


1.3. Scientific Theories 


Factor analysis, like all statistics, is a branch of applied mathematics. 
It employs observed data, which necessarily include discrepancies, and 
hence may lead to several different mathematical formulations of a given 
problem. In general, an applied mathematical theory consists of the follow- 
ing parts: 

SCIENTIFIC THEORY IN A GIVEN FIELD 
1. Observed data (with discrepancies) 
2. Mathematical theory 
Postulates 
Definitions 
Theorems 
3. Verification with data 
Any such applied science involves some 
plus a set of observations, and, finally, 
matical theory with the original data. Ir 
ics there may be a number of mathemat 
nomena in a satisfactory manner. A m 
between a mathematical theory and o 
tered. When a theory has been Success; 
data, there is a tendency to accept thi 
scribing the observations. 
Furthermore, 


mathematical theory as its basis, 
the check of the partieular mathe- 
1 all problems of applied mathemat- 
ical theories which explain the phe- 
isunderstanding of the relationship 
bserved data is frequently encoun- 
fully employed in describing a set of 
5 law as the only correct one for de- 


it is sometimes inferred that nature behaves in precisely the way 
matics indicates. As a matter of fact, nature never does behave in 
this way, and t| 


here are always more mathematical theories than one whose results 
depart from a given set of data by less than the errors of observation. 


The danger is always when a theory has been found to be convenient and effective 
over a long period of time, that people begin to think that nature herself behaves 
precisely in the way which is indi 

the belief that it is so may close 
impedence to progress in the 
around us.* 


* G. A. Bliss, “Mathematical Inte 


rpretations of Geometrical and Physical Phenome- 
na,” Amer. Math. Monthly, 1933, pp. 472, 477. (Italics ours.) 
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The above observations may be illustrated in several different fields. One 
of the simplest cases arises in the problem of surveying a small tract of land. 
For this purpose either of two mathematical theories, plane or spherical 
trigonometry, may be applied. Thus, in surveying a city lot the results by 
either theory would be equally satisfactory, and the engineer would prefer 
the plane theory because of its greater simplicity. In this instance, how- 
ever, there is no doubt as to the greater accuracy of the spherical theory 
since the earth is essentially spherical. 

In the field of astronomy there are two theories describing the solar sys- 
tem. The Ptolemaic and Copernican theories, with suitable modification of 
the former, describe the motions of the planets with equal accuracy. “There 
is really no advantage for either of these theories as compared with the 
other, as far as their adaptability to explain numerically the facts of the 
solar system is concerned. The Copernican theory is, however, much the 
simpler geometrically and mathematically. For this reason it has been 
adapted and developed until astronomers can predict coming celestial 
events with most surprising accuracy.” * 

Even the subject of geometry, which might seem to depend on a unique 
mathematical theory, can be described by means of many different theories. 
Thus, the physical configurations in a plane can be interpreted in the light 
of Euclidean geometry, Riemannian geometry, or various other types of 
non-Euclidean geometry. Therefore, the applied science of geometry can 
have several alternative theories as its basis. 

As in the foregoing illustrations there are different theories, or forms of 
solution, which may arise in the factorial analysis of a particular set of 
data. The usefulness of factor analysis as a scientific tool has been ques- 
tioned by some workers because of this indeterminacy. It should be evi- 
dent, however, that this is tantamount to indicting all applied sciences 
because they do not depend upon unique theories. 

Since the beginning of this century, psychologists and statisticians have 
developed several types of factorial solutions. The proponent of each sys- 
tem of analysis has urged its suitability for the interpretation of psycho- 
logical data. These varying points of view have been wittily described by 
Cureton as follows: 

Factor theory may be defined as a mathematical rationalization. A factor-ana- 
lyst is an individual with a peculiar obsession regarding the nature of mental ability 
or personality. By the application of higher mathematics to wishful thinking, he 
always proves that his original fixed idea or compulsion was right or necessary. In 
the process he usually proves that all other factor-analysts are dangerously insane, 
and that the only salvation for them is to undergo his own brand of analysis in 


* Ibid., pp. 477-78. 
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order that the true essence of their several maladies may be discovered. Since they 
never submit to this indignity, he classes them all as hopeless cases, and searches 
about for some branch of mathematics which none of them is likely to have studied 
in order to prove that their incurability is not only necessary but also sufficient.* 


The factor analyst must recognize the fact that methods other than his 
own must have some merit. This attitude can be expressed in a manner 
analogous to that employed by Barclay in his discussion of the various bid- 
ding systems in contract bridge.t The argument in the following two para- 
graphs is essentially that of Barclay with the phrases “bidding systems" re- 
placed by “types of factorial solutions" and “players” by "analysts." 

It is a positive fact that every one of the types of factorial solutions in 
extensive use today is an efficient one. When known thoroughly, a skilled 
analyst can apply any one of them to produce fine results in a desired situa- 
tion. The enthusiastic partisan who depreciates the efficacy of some other 
method and calls it a poor one thereby furnishes indisputable proof of one 
of two things—either his own sincerity or his own ignorance. Not just ig- 
norance of the details of the other method, but general ignorance. Any per- 
son of intelligence should know that it is highly improbable that any of the 
brilliant psychologists and statisticians who have worked on these problems 
for the last thirty years would produce and advocate a “rotten” system. 

The analyst who learns all the leading factorial methods and becomes 
familiar with them in actual use will be able to defend any method that 
someone may endeavor to ridicule. When some authority makes light of 
another method of analysis, it is quite likely that the man advocating a par- 
ticular system is extremely earnest in believing that he is offering his fol- 
lowers the very best procedures, but it may also be that the man has not 
given enough study to the other's method to realize its fine points as well 
as its deficiencies. Since no one factor method can cover all types of data 
which the analyst might like to describe, he should be familiar with all the 
methods so that he can select that one which is most sui 


table in a particular 
problem. 


The various types of factorial solutions correspond to the different mathe- 
matical theories in the description of a particular scientific problem. Five 
preferred forms of solution are presented in the text, any one of which may 
be selected by the analyst for dealing with a given body of data. The exposi- 


*Edward E. Cureton, “The Principal Compulsi n 
A pulsions of Factor-. t: 
Educational Review, 1939, p. 287. ea ees 


T Shepard Barclay, Win at Contract with Any Partner, 1933, pp. viii-ix. 
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tion of these methods and their comparison are the primary aims of the 
present text. 

The properties and statistical criteria underlying each of the preferred 
types are presented in Chapter V. These standards reveal the salient fea- 
tures of the preferred solutions, indicating their advantages and limitations. 
When an analyst has selected a particular form of factor solution for the 
description of a given problem, he should realize that there are other meth- 
ods which might afford an equally satisfactory interpretation of the data. 


CHAPTER II 


BASIC STATISTICS 
2.1. Introduction 


In this chapter certain elementary statistical notions are introduced. 
"These ideas are essential for much of the subsequent analysis and are there- 
fore presented in detail for the sake of rigor and completeness. The defini- 
tions are so formulated that they are applicable in any field, such as educa- 
tion, psychology, or commerce. Consequently, the formulas based on these 
definitions are general in their application. 

After the preliminary definitions of some elementary statistical concepts 
have been presented, the composition of a statistical variable is treated. 
Then a fundamental distinetion between two necessary parts of a factor 
solution, pattern and structure, is developed. A brief discussion of the statis- 
tical adequacy of a factor solution follows. The general arbitrariness of any 
scientific theory is exhibited here by the indeterminacy of factor solutions. 
Finally, methods of organizing data for subsequent analysis are presented. 


In illustrating these methods a set of psychological variables is employed. 


2.2. Statistical Variables 


Any statistical study involves a selection of a group of N individuals 
with a number of common attributes. In a particular investigation the 
group of individuals might consist of such objects as persons, schools, cities, 
or commercial enterprises. Various aspects of such individuals may be meas- 
ured and referred to as statistical variables. These will be designated by 
X;(j = 1,2,..., n), or more briefly j, where n is the number of variables 
in a set which is employed in describing the N individuals. Particular values 
of a variable X; for individual i will be represented by X;,. These will be 
called observed values inasmuch as they are measured from an arbitrary 
origin and by an arbitrary unit. 


The sum of all N values for a variable X; is written in the form 


N 


> ^T 


iz1 
but this may be put more simply in the form 
ZXjü " 


10 
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where it is understood that the summation extends over all values of the 
variable. This convention for the summation with respect to the number of 
observations of a particular variable will be observed throughout the text. 
Furthermore, the index i will be reserved explicitly to refer to the individu- 
als, that is, for the range 1 to N. The mean of the N values of X; is then 
given by 

IX; 


N 


M;= 
Derived values of each variable can be obtained by fixing the origin and the 
unit. When the origin is taken at the mean, a particular value 
tii = Xa — M; 


will be called a deviate. The standard deviation of a variable X;is defined by 


Yi si 
(2.1) s; = JI i 


where, of course, the summation is over the N values of i. By taking the 
Standard deviation as the unit of measurement, standardized values may be 
defined as follows: 


(2.2) z=. 


The set of all values z;; (i = 1,2,... , N) will be designated as a statistical 
variable z; in standard form. Obviously the variance (standard deviation 
squared) of z; is equal to unity. 

The product-moment coefficient of correlation for any two variables j and 
k is defined by 
Ert 
(2.3) Vik VEX VI: Vai 


or, making use of (2.1) and (2.2), 


E Lr jie: = Zzj£yi 


(2.37) NET Nojo, N 


The intercorrelations between all possible pairs of variables in a set con- 
stitute the basic data for factor analysis. It will be assumed here, as in all 
other cases where the product-moment correlation coefficient is employed, 
that it is most useful in case all regression lines are sensibly linear and all 
variables are normally distributed in the population and approximately so 
in the sample. 
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2.3. Composition of Variables 


The simplest mathematical expression for describing a variable in terms 
of several others is a linear one. This form is the one frequently employed 
in ordinary regression methods, although more elaborate expressions are also 
used for a small number of variables. For a large set of variables, however, 
2 simple approximation becomes almost imperative, and hence linear ex- 
pressions are used. Similarly, in factor analysis the simple linear form is 
taken to represent a variable z; in terms of a number of underlying factors, 
or hypothetical variables, which are taken to be in standard form for con- 
venience. À factor may be said to be in standard form if in the population 
its mean is zero and its standard deviation is unity. Furthermore, it is as- 
sumed that these properties also hold for a given sample. The factors are 
also assumed to be normally distributed. 

Several types of factors may be employed. Common factors are those 
which are involved in more than one variable of a set. Two instances of 
common factors arise: a general factor, present in all variables, and a group 
factor, present in more than one but not in all variables. A factor which 
appears exclusively in the description of a particular variable is said to be 
unique. Common factors are necessary in order to account for the inter- 
correlations of the n variables. Each unique factor represents that portion 
of à variable not ascribable to its correlations with other variables in the set. 

mploying the notation Fi, Fs, .. . , Fm for m common factors, and Ui, 
Us,..., Un for the unique factors, the complete linear expression for any 
variable z; (j = 1, 2, 3, .. . , n) may then be written in the form 


(2.4) 23 = ajF1 + aF: + aifi +... + Qim m + a;U;, 

where the prime is used to indicate the theoretical form of the observed 
variable. For simplicity the prime will usually be dropped hereafter in the 
linear expression for the variable. There are, of course, n equations of this 
form—one for each of the n variables. Equation (2.4) might be written 


explicitly for the value of variable z; for a particular individual 7 (i = 1,2, 
EE Yos dies 


(2.5) £j = aari + aF + afai t nn H GjsF mi + a;U j: 
In this expression it is assumed that there are N value 
tors, corresponding to the N individuals of the sample. The basic problem 
of factor analysis is to determine the coefficients, aj, . . 
mon factors. This determination is made by the va 
in Parts II and III, employing the observed intere 
ables. After these coefficients have been determi 
individuals can be evaluated by the methods of C 


s of each of the fac- 


b +++ djs, Of the com- 
rious methods described 
orrelations of the n vari- 
ned, the factors for the 
hapter XII. 
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The contributions of the factors to the total variance of any variable z; 
will next be shown. To express the variance of z; in terms of the coefficients 
of the factors, square both sides of (2.5), sum over the N values of the re- 
spective variables, and divide by N, and obtain, upon expanding, 

Za: _ SF; Z 
NW 


2 
2 


5 2. 
d 1 nU I 2h 


N 


FR, 
N ta 


LPF; DE mi iF mi 
+ 2 (anan N Tos. 0; maid jm — 


DFU: 
ZF: 
+ aja; N Pb... aj; 


ZFU;j; 
N š 


Now, since the variance of a variable in standard form is equal to unity, 
and all variables (including the factors) are assumed to be in standard form 
for any sample, the last equation may be written 


1 = 63, = aj, + ah +... +43, +a? + 2(ajnayerr,r, 


Se ai QjmQ,7F,,0;) " 


(2.6) 


If the factors are uncorrelated,* this expression simplifies to ^ 
(2.7) " 1-2ec5-ahcahc...-ctah-cad. 


The terms on the right represent the portions of the unit variance of z; 
ascribable to the respective factors. For example, a?, is the contribution of 
the factor F; to the variance of z;. The total contribution of F» to the vari- 
ances of all the variables is given by 


aj; + ah +a +... dr ah, 

n 
and, in general, the total contribution of a factor F, is defined to be >> aj, " 

jul 
For any variable z; the sum of the squares of the common-factor coeffi- 
cients is known as its communality h?. The last term in (2.7), which is the 
contribution of the unique factor, is called the uniqueness of z; and indicates 
the extent to which the common factors fail to account for the total variance 
of the variable. The uniqueness of a variable may be separated into two 
portions due to the selection of the variables and their measurement. If 
additional variables are added to a given set, their correlations with the 
latter may make it necessary to postulate further common factors. For a 


* The case of correlated factors is treated in Chap. XI. 
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fixed set of variables such potential linkages of any one of them with others 
can be expressed only as a portion of its uniqueness. This portion is termed 
the specificity of the variable. The remaining portion of the uniqueness may 
be ascribed to imperfections of measurement. The consistency with which 
a variable measures whatever it measures is called its reliability. The re- 
liability of a variable may be expressed by the correlation of repeated meas- 
urements or of parallel forms of the variable. In either case, the two evalu- 
ations will be represented by z; and zy, and their correlation rj wil be 
called the reliability of the variable. 

When the unique factors are conceived as decomposed into the two types, 


described above, the expression (2.4) for any variable may be written in 
the form 


(2.8) Z = [5122 + [5321 + ET + Qim? m + b;S; + cjT; 
(212,..,9). 


Here S; and T; are the specific and unreliable factors, respectively, and b; 
and c; their coefficients. Now, if the variance of 2; is obtained in terms of 


the coefficients of (2.8) and compared with the expression (2.7), it is evi- 
dent that 


(2.9) aj = + c?. 


From the preceding analysis it is apparent that the total unit variance 
of a variable may be resolved into three portions, attributable to common, 
specific, and unreliable factors. The first portion has been called the com- 
munality, while the last two portions of the variance of any variable have 
been named specificity and unreliability. On combining the specificity and 
unreliability, the uniqueness is obtained; or, again, the sum of the commu- 
nality and specificity gives the reliability of a variable. A recapitulation of 
these ideas "AY be expressed in the following form: 


b} = specificity , S; = specific factor , 
cj = unreliability , T; = unreliable factor j 
(20) | h? = Ya. = communality , F, — common factor , 
a=1 - 


i = b} + d = uniqueness = 1 — hi, 
k +b = reliability = 1 — È} = Tijs 

By factorial methods the communality n 
of variables are obtained. The uniqueness 
into the Specificity and unreliability, 


and the uniqueness a? of a set 
of each variable may be split | 
but this is independent of the factorial 
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solution and follows such a solution. If the reliability r; of a variable z;is 
known (it may be obtained by experimental methods), then the unreli- 
ability may be obtained by means of the equation 


(2.11) g=] = nz. 


Then, of course, knowing the unreliability, the specificity is given by the 
“Pythagorean” relation (2.9), or 


(2.12) b= a? —c?, 


where the uniqueness a? is known from the factorial solution. From the 
last expression in (2.10) it follows that 


(2.13) h =ru — OF Sry. 


In other words, the communality of any variable is less than or equal to 
the reliability of the variable, and equals the reliability only when the 
specificity vanishes. 
An index of completeness of factorization may then be expressed in the 
following form: . 
100}; 
wR 


communality 
reliability 


(2.14) H; = 100 
This index shows the percentage of the reliability variance accounted for 
by the common factors. If it is desirable to find the total communality, 


> h}, instead of the separate communalities for each variable, and also the 
j=1 
total specificity, then the complete index of factorization, which is defined by 


(2.15) H=SH;, 
j=1 


may be obtained. The index H plays the same role for the total set as H i 
does for the individual variable z;. 

The index H; is always less than 100 and approaches 100 only when the 
specificity b? vanishes. Any method of analysis for determining the coeffi- 
cients a;, should not be carried to the point where no specificity is present 
when it is known that some specificity exists. 

Some workers may not care to assume specific or even unreliable factors 
as indicated by (2.8). In this case the factors S ; and T; are not postulated, 

_ and the number m of statistically significant common factors may be less 
than, or equal to, the number of variables n. In the opinion of the authors, 
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however, the hypotheses of factors indicated by (2.8) appears most tenable 
even for variables which appear to describe a set of objects very completely 
and with great precision. 


2.4. Factor Patterns and Structures 
The factor problem will now be dealt with more generally, and for this 


purpose correlated factors will be postulated. The uncorrelated factors may 
then be considered as a special case of these. A set of equations of the form 


z, = ayF, Tay, + e + au F, + a,U, 
(2.16) Zz = Ay Fy + as +... + amf m T a,U, 


Za = Only Hanka H. o au F aU, , 


expressing 7 variables in terms of m common factors and n unique factors, 
will be defined as a factor pattern, or, more briefly, pattern. As indicated 
above, the common factors F, (s = 1, 2,..., m) may be correlated or un- 
correlated, but the unique factors U; (j = 1, 2,..., n) are always as- 
sumed to be uncorrelated among themselves and with all common factors. 
In the linear description of a given variable any number of common factors 
u(u € m) may be involved. This number is called the complexity of the 
variable. 

Factor analysis yields not only patterns but also correlations between 
the variables and the factors. A set of such correlations will be defined as 
a factor structure, or merely a structure. Both a structure and a pattern are 
necessary in order to furnish a complete solution. The functional relation- 
ships between the elements of a structure and the coefficients of a pattern 
will now be shown. 

Multiplying any one of equations (2.16) by the respective factors, sum- 
ming over the number of observations N, and dividing by N, produces 


Tür, = a Tages, T... aatre, +... + Gint PF 3 
T.r, = Qytryr, + aj +... + GaTE, H... H Bj rr. » 
(2.17) 
Tag, = Gj ; 
5B, = Qar, d Apter, +... + aj +... + TON 
a +... ET p, +... + dim, 
and 


(2.18) Tru; — Qj. 
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Equation (2.18) shows that the elements r.;v; of a factor structure are 
the coefficients of the unique factors in the pattern. When no confusion 
can arise, the table of correlations of variables with common factors only, 
i.e., the table of r.,r,, will be referred to as the factor structure. The ma- 
trix formulation of the relationship between a fector pattern and structure 
is given in Appendix B.11. 

In a numerical problem equations (2.17) may be used to evaluate the 
correlations of the variables with the factors when the coefficients of the 
pattern and the intercorrelations of the factors are known. More frequent- 
ly, however, these equations are used to obtain the values of the pattern 
coefficients when the correlations between variables and factors and the 
correlations between the factors themselves are known. 

Formally the equations (2.17) may be considered as constituting n sets 
of m linear equations in the unknowns aj (j = 1, 2,..., n; s= 1, 2, 
...,m), with the left-hand members as known quantities. It is then pos- 
sible to solve* for the m coefficients a;, in each of the n sets, that is, for each 
value of j. Here it has been assumed that the elements of the factor struc- 
ture and the correlations between factors are known, and so the coefficients 
of the factor pattern may be obtained. The result is 


(2.19) Ga = 71 


where A is the determinant of coefficients of the a;, and A, is the determi- 
nant obtained from A by replacing the elements of the sth column by the 
elements TejF yy Tejhyy e ri Tee It has thus been shown that a pattern can 
be constructed from a known structure. 

It is interesting to note} that the values a;, in the above solution turn 
out to be regression coefficients, so that equations (2.16) are in the form of 
regression equations with the errors of estimate included. These errors are 
represented by the unique factors. A plus-or-minus sign may be placed in 
front of the a; because the unique-factor coefficients in a pattern are usually 
obtained, after all the other coefficients are known, in such a manner as to 
bring the variance of each variable up to unity. This procedure involves a 
square root and thus leads to the indeterminacy of sign. 

Since the unique factors are of minor interest in factor analysis, it is con- 
venient to write that portion of a factor pattern which involves only the 
common factors. For similar reasons, a table of coefficients of the common 
factors may be referred to as a factor pattern, when there is no ambiguity. 


* Maxime Bocher, Introduction to Higher Algebra, 1935, p. 43. See also Appen. G.1 
for a simplified method of solving a set of simultaneous equations. 


1 See Appen. B.2. 
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From equations (2.17) it is apparent that the elements T;r, of a struc- 
ture are generally different from the coefficients aj, of a pattern. In case 
the common factors F, are uncorrelated, that is, rz r, = 0(s # t), then equa- 
tions (2.17) reduce to 


(2.20) Tug, = Die FG 2; caw pg oS Qh s mn) 


Thus, only in the case of uncorrelated factors are the elements of a structure 
identical with the corresponding coefficients of a pattern. In an analysis in- 
volving uncorrelated factors, æ complete solution is furnished merely by a 
factor pattern inasmuch as the correlations of the variables with the factors 
are given by the respective coefficients. 

As already indicated, both structure and pattern should be produced in 
making a complete factor analysis. The structure reveals the correlations 
of variables and factors, which are useful for the identification of factors 
and for subsequent estimates of the latter. The pattern shows the linear 
composition of variables in terms of factors in the form of regression equa- 
tions. It may also be used for reproducing the correlations between vari- 
ables to determine the adequacy of the solution. In comparing different 
systems of factors for a given set of variables, patterns are again useful. 


2.5. Statistical Basis for Adequacy of Solution 


The manner in which a set of factors accounts for the intercorrelations in 
a set of variables will be considered next. The variables are assumed to be 
linearly composed of the factors as given by the pattern (2.16). Since the 
factors are assumed to be in standard form, the correlations between vari- 
ables may then be reproduced from the factor pattern by the following pro- 
cedure: multiply any equation (2.16) of a variable by the equation of an- 
other, sum over the number of individuals N, and divide by N. For any 


two variables z;, z; (j, k = 1, 2,..., n) their correlation may be written 
in the form* : 


Tik = aiaa + ajar + aizak +... + O jim km 


+ (anar + araj) Tr r, + (ajia4s + Qs) P,P, Aus 
(2.21) + (antis + aajs)rr v, + (apara + Os js) p, r, +... 
F (Aptim + dian) Trey A... H apare u, + uj eu; +... 


+ Qima epu, + imi ru; + jAK jo, , 


* Holzinger and Harman, “Relationshi 


ps between Factors Obtained f. i 
Analyses,” Journal of Educational Psycho dir ion 


logy, 1937, p. 324. 
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where the reproduced correlation (computed from the pattern) is written 
rj, to distinguish it from the observed correlation r;;. This distinction will 
be made throughout the text. 

The unique factors have been assumed to be uncorrelated with the com- 
mon factors and among themselves, hence frv; = Tr,v, = fuu, = 0. If 
the common factors are uncorrelated, equation (2.21) simplifies still further. 
The correlations rz,z, (s # t; s, t = 1, 2, . . . , m) are then zero, and every- 
thing below the first line of the equation vanishes. For the case of uncor- 
related common factors, any correlation is thus reproduced from the factor 


pattern by an equation of the following form: 


(2.22) Tik = Quin + Qj ka Teu QjmAkm 
G = k; j,k =1,2,...,0). 


This expression is merely the sum of the products of corresponding pattern 


coefficients of the two variables correlated. 
Equation (2.22) may be represented, perhaps more simply, by use of 
matrix notation.* Let the matrix of coefficients of the common factors of 


the pattern be represented by A, that is, 


Qu AQ Q Aim 
Ag, Qz Q3 Gam 
(2.23) A= " 
Qj Aja djs ... Qj 
Qni An Ang +--+ Anm 


which will be referred to as the pattern matrix. Then the product of A by 
its transpose A’, i.e., the matrix obtained from A by interchanging its rows 
and columns, is the matrix of reproduced correlations Rt. This result may 


be put in the formt 
(2.24) Rt = Aa’, 


* For an elementary account of the theory of matrices which is essential for the pres- 
ent text see Appen. A. 

f Equation (2.24) has been called “the fundamental factor theorem" by L. L. Thur- 
stone (Vectors of Mind, 1935, Chap. II). In the case of correlated factors, which will be 
treated in Chap. XI, the expression corresponding to (2.24) is 

Rt = Ada’, 
where $ is the matrix of the intercorrelations of the common factors. This relation was 
first presented, in expanded form, by Holzinger and Harman, op. cit. More recently, it 
was formulated in matrix notation by Tucker, "The Role of Correlated Factors in Factor 
Analysis," Psychometrika, 1940. 
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or, writing the matrices in detail, as follows: 


] 
hi Wy see Wm ces Tin 
, 
i e ss th evr & 
| . 
/ , k m 
Ti Tig +--+ MUG e. Tijn 
2 
Vie: dE sss The cie TE] 
Rt 
i] ' 
| ün Q Alm 
|an az dom Gi. 05 ay Ant 
|. é% mE | ar Ax ajo an? 
= BE 
| Gi Qj Gud dessus 
| 
E . -| Vite Bix sce Wie aux tix 
| 
|| ani an2 Anm | 
= A s A’. 


The correlations (2.22) are merely elements in the matrix Rt. The diagonal 
elements hj (j = 1, 2, . . . , n) are the communalities of the respective vari- 
ables and may also be obtained by setting j = k in equation (2.22). It may 
be noted from (2.21) that the algebraic signs of all the entries in any column 
of a pattern matrix A may be reversed without affecting the reproduced 
correlations. 

Evidently the factor problem is concerned with the correlations of a set 
of variables. These correlations are reproduced, when a pattern of the form 
(2.16) is assumed, by means of the coefficients of the common factors. The 
diagonal elements of R should also be reproduced from this portion of the 
factor pattern. Thus if, in a matrix of observed correlations, numbers ap- 
proximating the communalities are put in the diagonal, the factor solution 
will be of the form (2.16) and the observed data (including the diagonal 
entries) will be closely reproduced by equation (2.24). On the other hand, 
if unity is in each diagonal cell of the observed correlation matrix, the factor 
solution necessarily would involve only common factors in order that equa- 
tion (2.24) may reproduce ones in the diagonal. In this case there is obvi- 


YW Axi 
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ously no allowance made for unique factors. If other values between the 
communalities and unity (such as reliabilities) are employed, then the form 
of solution would involve common and unreliability factors, but the specific- 
ity would be included in the common-factor variance. From these consider- 
ations it is clear that the values put in the diagonal of the observed correla- 
tion matrix determine what portions of the unit variances are being fac- 
tored. 

The general problem of relating the reproduced to the observed correla- 
tions, regardless of the diagonal entries, will be considered next. The correla- 
tions reproduced by the factor pattern, as given most generally in (2.21) or 
for the case of uncorrelated factors in matrix Rt, should not agree exactly 
with the observed correlations because allowance must be made for sampling 
and experimental errors. It is a commonly accepted scientific principle that 
a theoretical law should be simpler than the observed data upon which it is 
based, and hence discrepancies between the law and the data are to be ex- 
pected. In the case of factor analysis, functions (the correlations r/,) of the 
assumed linear composition of variables should be expected to vary some- 
what from the observed values. 

After a factor pattern has been obtained, its adequacy as a description 
of the intercorrelations of the variables is determined by “removing the 
factors." This is done by forming the correlations from the pattern as given 
by equation (2.21). These values are then subtracted from the correspond- 
ing observed correlations, and the resulting differences are known as residual 
correlations. Such residuals are defined by the following formula: 


(2.25) Fie = Th), 


where r;, is the observed correlation and r/; is the correlation reproduced 


from the pattern. 
In case the common factors are uncorrelated, it has been shown that the 


lengthy expression (2.21) for reproducing the correlations reduces to the 
simple form (2.22). The residuals then reduce to the form 


(2.26) Pj = Tj — (nan + ajar +... + Ajm@km) . 


The question then arises as to how nearly the correlations reproduced 
from a factor pattern should fit the observed ones.* The agreement may 


* Up to the present time a good standard for “when to stop factoring” has not been 
developed. This problem is discussed in Chaps. VI, VII, and VIII. 


NO 4 69 
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be judged by the size and distribution of the residuals 7;;. The magnitude 
‘of the residuals should, of course, be approximately zero. When all com- 
mon factors have been removed in forming the residuals, then no further 
linkages between variables exist. It might, therefore, be expected that the 
distribution of residuals would be similar to that of a zero correlation in a 


sample of equal size. The standard error of such a zero correlation is given 
by the formula 


el om VM 
N—1 


or, since N is usually large, 


(2.27) PARES x 


A standard for judging adequacy of fit then may be taken to be 


(2.28) «€ 


where g; is the standard deviation of the series of residuals.* This standard 
is a tentative one since it depends only on the size of the sample, whereas 
experiments have shown that the size of residuals depends also on other 
characteristics, especially the number of variables. More exact sampling 
error formulas will be developed in Chapter VI. 

On the basis of the size of the sample alone, the following conclusions from 
the criterion (2.28) may be drawn. If c; is appreciably greater than 1/VN, 
it may be concluded that there are further significant linkages between 
variables, and a modification of the form of solution is required. In case c; 
is considerably less than 1/ VN, it would appear that unjustified linkages 
between variables have been included in the solution. When the standard 
deviation of the residuals is just below that of a zero correlation, the solu- 
tion may be regarded as acceptable in the light of the above standard. A 
standard equivalent to (2.28) has been given by Kelley, and substantially 


the same basis for judging the adequacy of fit of a factor solution has also 
been given by Thurstone.t 


x * 
* An alternative value for c2, taking degrees of freedom into account, is given by the 
2:5 
formula ES - mn — m X 1) i 


T Truman L, Kelley, Essential Trails of Mental Life, 1935, p. 12, formula CO 
Op. cil., p. 147. 
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2.6. Indeterminacy of Solution 


In any scientific field the observed phenomena can be described in a great 
variety of ways which are mutually consistent. The choice of a particular 
interpretation must then depend upon its utility. This arbitrariness or in- 
determinacy of description has been recently emphasized by F. R. Moulton 
in comparing various approaches to the measurement of the velocity of 
light. In conclusion he says: 

All this illustrates Poincaré’s thesis that every set of phenomena can be interpret-«t 
ed consistently in various ways, in fact, in infinitely many ways. It is our privilege 
to choose among the possible interpretations the ones that appear to us most satis- 
factory, whatever may be the reasons for our choice. If scientists would remember 
that various equally consistent interpretations of every set of observational data 
can be made, they would be much less dogmatic than they often are, and their 
beliefs in a possible ultimate finality of scientific theories would vanish.* 


The basic phenomena for factor analysis are the correlations of a set of 
variables. An explanation of these correlations can be made in the form of 
a factor solution in an infinite number of ways.t This indeterminacy is ex- 
hibited by the fact that various systems of factors may be selected, yielding 
solutions which reproduce the observed correlations equally well. In order 
to remove this indeterminacy, certain statistical criteria may be introduced. 
A number of such standards are given in Chapter V which lead to a few 


types of preferred factor solutions. 


2.7. B-Coefficients 

One of the fundamental problems in factor analysis is the selection of the 
descriptive variables. These should be chosen so as to measure the aspects 
of the group which are significant for a particular problem, i.e., the set of 
~ variables should be valid as a whole. This criterion implies that generally 
the intercorrelations will be positive throughout, or else can be put in this 
form (see 5.7). The variables should also. be selected in such a way as to 
make possible the identification of the underlying factors. For this purpose 
it is necessary to choose at least three or four variables of a kind which ap- 
pear to measure the same factor. The variables in each group of this type 
should not be mere parallel forms but should be distinct in content. This 
hypothetical design of the variables is tested by the factor analysis, which 


* “The Velocity of Light,” Scientific Monthly, 1939, p. 484, a 


t For proof see Appen. B.3. It is shown there that this indeterminacy corresponds to 
the infinite number of rotations from one system of reference axes to another. 
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gives evidence for retaining or rejecting the original groupings of the vari- 
ables. 

In actual practice the above groupings are usually based on previous re- 
search in which some of the factors have already been identified. The design 
can then be extended to include other groups of variables used to identify 
additional faetors. In some cases it may be desirable to take a portion of a 
previous design and add new variables to obtain more refined measures of 
the factors already identified. The success of the factorial analysis depends 
in a large measure on the skill with which the variables in such groups have 
been selected. 

If no design ean be predetermined when the set of variables is com- 
piled and the matrix of correlations obtained, the grouping of the variables 
may be indicated by an objective procedure. This method is based on the 
assumption that the variables of a group identifying a factor should have 
higher intercorrelations than with other variables in the set. To this end, 
the B-coefficient, or coefficient of belonging, may be employed. This coeffi- 
cient is defined as 100 times the ratio of the average of the intercorrelations of a 
subset or group of variables to their average correlation with all remaining 
variables. The group of variables for which B is computed is referred to as 
the “argument of B,” while the remaining group of variables is called the 
‘complementary subset." The number of variables in the argument of B 
increases, beginning with two, at successive stages, and the value of the 
B-coefficient at each stage may vary accordingly. It is evident that the 
value of B for a given group of variables is independent of the order in 
which the variables are involved in the argument of B. On the other hand, 
the time when a particular variable enters into the argument of B is relevant 
to the value of B. For example, the B-coefficient of zi, 23, 24, which is writ- 
ten B(z, zs, 23), or, more briefly, B(1, 3, 4), has the same value no matter 
what order the three variables assume. The value of B may differ, however, 
depending on the sequence in which another variable, say z», is added. For 
concreteness, the value of B(1, 3, 2) is usually different from B(1, 3, 4, 2). 
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To define the B-coefficient more rigorously and thus obtain a formula for 
its computation, the following notation will be employed: 


n — total number of variables in a set, 
u = the subset of variables in the argument of B, 
p — number of variables in the subset u, 

Jj, k = indices ranging over u, 


cu = the complementary subset of variables, that is, those 
variables of the original set which are not in u, 


a = index ranging over cu, 


S= bP Tix = sum* of intercorrelations of variables in the 
(2.29) jck 
subset u, 


Fa s Tia = sum} of correlations of variables in u with 
Ja 


those in cu, 


2 
taken two ata time;f here, the number of correlations 
involved in the sum S, 


im .Jp-L. number of combination of p elements 


p(n — p) = number of correlations involved in the sum T. 


* The symbol Drie means the sum of all the correlations r;, for the specified range 
J«k 

of j and k under the condition that j is always less than k. In the present section the 

summation extends over all the p(p — 1)/2 combinations j « k of the p numbers taken 

two at a time. An illustration of such a sum in evaluating B(1, 2, 3, 4) may be written 


fully as follows: 
S = re = mu + Tis + ra + Ta + Tu + růu. 
j«k 


t An example of a sum T for a total set of 6 variables in the evaluation of B(1, 2, 3, 4) 


follows: 
p» 2573 = fis + Tie + Tos + Tos + Tus + 736 + Tas + Tee. 
ja 


In this example, n = 6, u = (1, 2, 3, 4), p = 4, j, k range over 1, 2, 3, 4, cu = (5, 6), 
and a ranges over 5, 6. 


i The symbol () is used throughout the text in place of the conventional C7 to desig- 
v 


nate the number of combinations of n things taken r at a time. 
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The B-coefficient of the subset of variables u is then defined by 


EA 
(i) 200(n — p)S . 


lan 5) d 


(2.30) B(u) = 100 


For purposes of computation of B-coefficients it is perhaps more con- 
venient to write the formula for T in another form. Since the sums of the 
correlations of the respective variables with all others are usually obtained 


at the outset of any analysis, they may be used in the evaluation of T for 
any B-coefficient. Thus, 


(2.31) P= > re = >t — 2 > ra= Dore — 28, 
jia 


Je j«k ive 


where the index e ranges over the entire set of variables.* The symbol 


> stands for the sum of the correlations of the variables in u with all 
ixe 
other variables. Then to obtain T it is necessary to subtract twice the inter- 
correlations of the variables in u. 

Another aid in the computation of the B-coefficients is the sum of the 
correlations of the last variable added to the group with the preceding ones. 


Letting | denote the last variable added to the group, the proposed sum 
may be written 


(2.32) L= ra, 


jæl 


where the summation extends over the variables in the subset u. If a sub- 
script p is appended to the sums L, S, and T to designate their values for 


* An illustration of formula (2.31) for the example of the last footnote follows: 
T =J ri = 2 ties 
ET j«k 
= (ria + ris F Tia + Tis + Tie) + (ra + oras + rs + Tas + ra) 
F (ra + ra + Tss + Ts + ra) + (ra + re tra + ri + Te) 
—2(ris + ris + Ti + T + Tea + Taa), 
= Tis + Tie + Tas + Tas + Tas + Tas + Tas + Ta. 


Although this formula for T seems to be more complex than that given in (2.29), its 
application to actual data is more effective than the latter. . 
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p variables in the argument of B, then successive values of S and T may 
be obtained by means of the following recurrence formulas: 


(2.33) Sy = Sia ds 

and 

(2.34) T, = Tpit ra — 2L,, 
exl 


where >) ra is the sum of the correlations of J with all the other variables 
exl 
in the set. 

The value B(u) = 100 indicates that the variables of the subset u have 
the same average intercorrelation as their average correlation with all re- 
maining variables. Such variables would not be regarded as "belonging to- 
gether,” since they belong just as much with the other variables of the total 
set. As a tentative standard of belonging together, the B-coefficient of a 
group of variables should be at least 130. 

The B-coefficients are used to sort the variables on the basis of their inter- 
correlations. The grouping is begun by selecting the two variables which 
have the highest correlation. To these is added the variable for which the 
sum of the correlations with the preceding is highest. This process is con- 
tinued, always adding a variable which correlates highest with those already 
in the argument of B, until a sharp drop appears in the value of B. When 
this occurs, the last variable added is withdrawn from the group. Another 
variable may be inserted in its place, but, if the drop in B is still large, it 
should be withdrawn. Thus a group of variables that belong together is 
determined. Then, excluding the variables that have already been assigned 
to such a group, the two others which have the highest remaining correla- 
tion are selected to start another group. To these variables are added others, 
exclusive of those that have already been assigned to groups, until a signifi- 
cant drop appears in B, at which stage another group is formed. It is de- 
sirable to start each new group with a B-coefficient as large as possible so 
as to have clearly defined groups. For this purpose it may be necessary to 
obtain the B-coefficients for more than one pair of variables without com- 
pleting the groups. The pair yielding the highest B value may then be used 
to introduce the next group. This process is continued until all variables 
have been assigned to groups or else do not fit into any group. 

At the present time no sampling error formula is available by which to 
Judge the significance of the difference between two successive values of B. 
One basis is to use all possible knowledge about the variables. It is a sound 
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scientific principle to use all the facts about the data of an experiment. The 
original hypothesis about the nature of the set of variables may thus aid in 
deciding whether a drop in B is "significantly" large. More objectively, 
some properties of the B-coefficient will now be presented. 

Since the B-coefficient is the ratio of two averages, its properties may be 
studied by means of them. The average of the intercorrelations of the vari- 
ables u (the numerator of B) tends to decrease as the number of variables 
in B increases, since the variables are added on the basis of highest correla- 
tion with those already in the argument of B. Similarly, the average of the 
correlations of the variables u with those in cu (the denominator of B) tends 
to decrease with an increase in p. The decrease in the numerator is relative- 
ly greater, however, than that in the denominator. 

To the numerator, which usually consists of a small number of correla- 
tions, are added a few additional correlations which are lower in value than 
the others and thus steadily decrease its value. On the other hand, from the 
large number of correlations in the denominator a small number of the larger 
values is taken away. The value of the denominator is decreased, but not 
so noticeably as that of the numerator. Thus the B-coefficient decreases, in 
general, as more variables are added to its argument. 

An exception to this may occur with the addition of a variable to the 
subset u which has relatively high correlations with the preceding variables 
but a low sum of correlations with those in cu. In this case the decrease in 
the numerator is relatively smaller than that in the denominator, and B in- 
creases. Similar reasoning accounts for the fact that a variable can be re- 
jected from a group because of a large drop in the value of B and then ap- 
pear in the group later, after several others have been added to the subset u. 

As the number of variables in the argument of B increases, the decreases 
in the above averages become less and these averages tend toward stability. 
A consequence of this is that an actual difference between two successive B 
values has a greater relative significance as the number of variables p in- 
creases. 

It is convenient to refer to the variables which are found to belong to- 
gether by the B-coefficients as constituting a group G, (s = 1, 2,..., m). 
The serial numbers s may be assigned in order to the groups of variables 
which best measure the respective factors (other than a general factor). 
Thus, the variables at the head of the set that are found to belong together 
may be given the group number 1 and be said to belong to G;. The factor 
which is predominantly measured by this group of variables may be sym- 
bolized by F;. Similarly for the other groups of variables, the final subset 
forms the group Gm, and the factor it measures is denoted Fm. If a solution 
involves a general factor Fo, the variables which measure it (i.e., all the 
variables) may be referred to as constituting a group Go. 
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2.8. Illustrative Example 

In order to illustrate and clarify factorial methods, a numerical example 
is given here. This example is employed throughout the text whenever pos- 
sible. Various factor solutions will be obtained for the same data, and com- 
parisons made of different systems of factors. 


TABLE 2.1 


MEANS, STANDARD DEVIATIONS, AND 
RELIABILITY COEFFICIENTS 


Standard Reliability 
me Mean | Deviation | Coefficient 
, 9i Tis 

29.60 6.90 -756 
24.84 4.50 .968 
15.65 3.07 .544 
36.31 8.38 .922 
44.92 11.75 .808 
9.95 3.36 .651 
18.79 4.63 .754 
28.18 5.34 .680 
17.24 7.89 .870 
90.16 23.60 .952 
68.41 16.84 .412 
109.83 21.04 .937 
191.81 37.03 .889 
176.14 10.72 .648 
89.45 7.57 .507 
103.43 6.74 .600 
7.15 4.57 s125 
9.44 4.49 .610 
15.24 3.58 .569 
30.38 19.76: .649 
14.46 4.82 . 784 
27.73 9.77 . 787 
18.82 9.35 .931 
25.83 4.70 .836 


The data employed for this purpose include a set of twenty-four psycho- 
logical variables (or tests), described briefly in Appendix B.1. In Table 2.1 
the means, standard deviations, and reliability coefficients are presented. 
"These statistics are essential for the factorial methods that are treated. 

The intercorrelations of these variables, which form the basic data for 
subsequent analyses, are presented in Table 2.2. The complete set of cor- 
relations could be presented in the form of a symmetric matrix with the 
elements in the principal diagonal omitted. Obviously only half of this ma- 
trix is necessary for its complete description, and it is therefore presented in 
this form for simplicity. The correlations for any variable j are located by 
reading across the row j to the diagonal and then down the column j. Then 


TABLE 2.2 
INTERCORRELATIONS OF TWENTY-FOUR VARIABLES 


Quooecme|. 3 
~g 


9 10 11 12 | 13 14 | 15 | 16 17 18 | 19 


21 


22 


23 | 24 


1424] .531 


A3 
463) 
509) 
.366, 


.528/4.751 


7.909,8 


3068. 156|5.666|7 .089|5.877|7 .581|5..443|5.079|6 .609/6 .009|6 .808/5.754|7 .755 


=I 


606 


7.693 


8.678/8.220 
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the sum of the twenty-three correlations of a variable j with all the others 
is given by the total of the entries in the row and column for test z;. This 
sum is represented by >> Tje, where j is fixed and e ranges over the twenty- 


nj 
four variables under the restriction e = j. The sum of the correlations for 


each variable with all others is also given in Table 2.2. 


2.9. Application of B-Coefficients 
To illustrate the method of grouping variables outlined in 2.7, the ex- 
ample of 2.8 will be employed. The analysis into groups is begun by select- 
ing the two tests—5 and 9—which have the highest correlation, namely, 
Uso = .723. The value of B(5, 9) is computed by means of formula (2.30) in 
: Table 2.3. The tests u, comprising the argument of B, are for this case z; 
and zs, and their correlation appears as the value of L and S, since there is 
only this one correlation in each of the sums. The value T' is given by 


T = Erse + Er, — 2rsy = 8.242 + 8.156 — 2(.723) = 14.952 3 


where the sums of the correlations are taken from Table 2.2. Then the 
value of the B-coefficient is 


200(n — p)S _ 4400(.723) _ o 19 
(p-)T - 1498 i 


B(5, 9) 


The form of computation indicated by Table 2.3 will be found very con- 
venient for machine calculation. In addition, the use of the sum L, defined 
in (2.32), and the recurrence formulas (2.33) and (2.34) will greatly facilitate 
the calculation of successive B values. 

To illustrate the use of these formulas, the value B (17, 18, 19, 15, 16) will 
be calculated, showing all details. Here, p = 5, and the last variable added, 
l, is 16. From definition (2.32) 


L= L; = Mrs = 117,16 + 718,16 + T1916 + 715,16 = 1.251, 
jz10 


the individual correlations being given in Table 2.2. The sum S, or Ss, may 
be obtained from the value S, by means of equation (2.33) as follows: 


S = Ss; = S, + Ls = 2.001 + 1.251 = 3.252. 


It should be noted that there are two S4 values but that one has been re- 
Jected as indicated by note (10). The next entry is merely 


200(n — p) = 200(24 — 5) = 3,800. 
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TABLE 2.3 
CALCULATION OF B-COEFFICIENTS 


| | | B(u)- 
u p L S |200(n—p) vd (p—1)T | 200(n—p)S | Notes 
| | (p-)T 


.193  .723 4400 | 14.952 14.952, 213 


1.341) 2.064) 4200 |.20.179| 40.358 215 
2.058 4.122 4000 | 24.245) 72.735 227 
2.950 6.378, 3800 | 28.039 112.150 216  |..... 
2.276 8.654 3600 | 32.165 160.825) 194 (1) 
2| .585 .585| 4400 | 10.373) 10.373 BAS: ases 
3| .920 1.505) 4200 | 16.117, 32.234 196 (2) 
3| .912 1.497| 4200 | 15.638] 31.276 201 (3) 
.. 4| 1.455 2.952 4000 | 20.312 60.936, 194 (4) 
(10, 12; 11,13,24).... 5 | 1.715 4.667) 3800 | 25.102 100.408 — 177 (5) 
(10,12,11,13,21).... 5 | 1.584 4.530 3800 | 24.750) 99.000 174 (6) 
(00,98). cus cn mtr 8] 2| .509 .509 4400 | 15.415 15.415| — 145 (7) 


.468 .468) 4400 | 12.637) 12.637) 103 (8) 
.708 1.176, 4200 | 16.530, 33.060 149 
.865 2.041) 4000 | 19.551) 58.653 189 |. wes 
1.189 3.230} 3800 | 24.779) 99.116, 124 (9) 


i | 

.448 -448| 4400 11.921| 11.921| 165 
1.130, 4200 16.311| 32.622 145 |... 
.926 2.056, 4000 | 21.068 63.204 130 (10) 
.871, 2.001, 4000 | 19.648| 58.944 136 | 
1.951 3.252, 3800 23.755| 95.020, 130 
1.530 4.782 3600 26.138| 130.690. 192 ees 


1.836 6.618, 3400 | 30.686| 184.116, 122 (1) 
| 
1.704, 6.486, 3400 | 30.423) 182.538 121 2) 


.509 .509 4400 15.415 15.415. 145 
-966 1.475) 4200 | 21.176 42.352 146  |..... 
.238, 2.713) 4000 | 26.306 78.918, 188 |... 
.623. 4.336 3800 | 31.280) 125.120. 182 .]|....- 
.652) 5.988 3600 | 34.784| 173.920) 124 | (18) 
-641) 5.977) 3600 | 34.607, 173.035! 124 ( 

| 


[p o MM | 
(20, 23, 22)... 
iU 23, 22, 21) 


2 

3 

4 

20, 23, 22, 21,24). . 5 
6 

6 


(20, 23, 22, 21, 24, 18). 
(20, 23, 22, 21, 24, 16). | 


pes pms pi jet 


NOTES ON TABLE 2.3 
(1) Test 23 is rejected because of 22 points’ drop in B for p —6. 
(2) Test 13 is rejected because 52 points’ drop in B seems to be too great even for p —3. 
(3) Test 11 is retained although it causes a drop of 47 points in B. The drop in B is not 
80 significant when p —3 as it is when p is larger. Furthermore, Test 11 is of the same 
general nature as Tests 10 and 12. 
(4) Test 13 is retained, although it was previously rejected from the group. After Test 


11 was put in the group, Test 13 seemed to belong together with the others, causing 
a drop of only 7 points in the value of B for p=4. 


(5) Test 24 is rejected because of 17 points’ drop in B for p=5. 
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(6) Test 21 is rejected because of 20 points’ drop in B for p=5. 

(7) Before continuing with the group which starts with Tests 20 and 23, another pair 
of tests will be tried to see if they yield a value of B greater than 145. 

(8) Tests 1 and 4, although having a lower correlation than the pair 20 and 23, produce 
a higher value of the B-coefficient. Hence the next group is started with Tests 1 
and 4. 

(9) Test 21 is rejected because of 15 points’ drop in B for p —5 and seemingly different 
nature of Test 21 from Tests 1, 2, 3, and 4. 

(10) Test 16 is rejected because of 15 points’ drop in B to see if some other test will cause 
a smaller drop. If some other test cannot be found which causes a smaller drop in B, 
then Test 16 will be retained in the group at this stage because a drop of 15 points 
for p —4 does not seem to be definitely significant. 

(11) Test 24 is rejected because of 10 points’ drop in B for p=7. 

(12) Test 22 is rejected because of 11 points’ drop in B for p=7. 

(13) Tests 18 and 16, although they had previously been allocated to another group, are 
put into the argument of B along with 20, 21, 22, 23, 24 to see if the latter group 
of tests must be extended to other tests in the battery. The drop in B in each case, 
along with the seemingly different nature of Tests 18 and 16, seems to warrant their 


rejection from this group. 


The sum T, or Ts, is given by formula (2.34) in terms of the preceding value 
Ts, which is in the row (17, 18, 19, 15) for p = 4, not in the row (17, 18, 19, 
16). Its value is 


T=T,=T.+ Sree — 2L; = 19.648 + 6.609 — 2(1.251) = 23.755, 


e#16 


where the sum of the correlations of Test 16 with all other tests is taken 
from Table 2.2. Then 
(p — 1)T = 4(23.755) = 95.020, 


and 


200(n — p)S | 3800(3.252 
B(17, 18, 19, 15, 16) = me wip - L -— 3 


Following the procedure outlined in 2.7, all the variables are grouped by 
means of B-coefficients. The groups G, (s — 1, 2, 3, 4, 5), as determined in 
Table 2.3, may be defined by 


6 = (1, 2, 3, 4), 
Gs = (5, 6, 7, 8, 9) , 
(2.35) G; = (10, 11, 12, 13) , 
G, = (14, 15, 16, 17, 18, 19) , 
Gs = (20, 21, 22, 23, 24) . 
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i i the original design of this set 
ouping by B-coefficients adheres to à wr 
of prea most of these tests had been used in factor bs Saige 
before and because the abilities they measured were quite well known. T ^, 
may not be true in general. In à number of published studies the B-coe : 
cients and the succeeding factor solutions failed to verify some postulate 
factors.* 


2.10. Correlations of Sums 


It is often convenient for subsequent analysis (e.g., Chaps. XI and XII) 
to consider a series of variables combined to form a single variable. Such 
new composite variables may then be put in standard form, and the correla- 
tion between two such variables obtained.t Thus it may be useful to com- 
bine the respective series of variables of each group G, (s = 1, 2,... m), 
so that in place of the original n variables there will be only m composites. 

The reduction to composite variables and the intercorrelations between 
such derived variables may be done in the following manner. Consider two 
SeriesS—2, 22; ... , 2, and Zr, Zu, . . . , 2g—consisting of qand Q variables, 
respectively. Each variable involves N values and is in standard form, i.e., 
has zero mean and unit standard deviation. Let the sums of these series of 
variables be denoted by the following composite variables: 


y=atat...+4, 
(2.36) AE vot aid 


the values of which are obtained by substituting the corresponding values 
of the observed variables on the right. It is apparent that these new vari- 
ables are also measured from their means as origin. Since different letters 


* Àn example of this appeared in the factor solution obtaine 
man, "Comparison of Two Factorial Analyses," 
paring the test battery, Professor Thurstone 
numerical reasoning, 


d by Holzinger and Har- 
Psychometrika, 1938, pp. 45-60. In pre- 


had postulated that “verbal reasoning, 
and space reasoning would be separate factors and that these would 
be different from verbal abstraction and visual imagery” (see L. L. Thurstone, “The 
Perceptual Factor,” Psychometrika, 1938, p. 11). Both his and the authors’ factor solu- 
tions cut across these predetermined gro 


upings that had guided the test construction 
and revealed some different factors. 


f Charles Spearman, “Correlations of Sums and Differences,” British Journal of Psy- 
chology, 1913, pp. 417-26. 


1 The variables Zt, Zit, « . . , Za Should not be interpreted as denoting parallel forms of 
some others. The roman subscripts are used here merel 


à y as a convenient way of desig- 
nating a second series of variables, distinct from the series 2, 22,..., Zo 
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are used to denote the two composite variables, no subscripts are necessary 


to distinguish them, and a single subscript is sufficient to represent the in- 
dividual values of these variables. Thus, 


(2.37) 


V; = Zig Zac... E 
Wy = Ait Ani F... + Zoi 


give the N values of the composite variables. 
The correlation between the composite variables is given by 


_ Zvw; 


(2.38) Tow Novo. 


To obtain an expression for ry» involving the correlations between the origi- 
nal variables, a formula for the standard deviation of a sum of variables is 


first required. By definition, 


Dv? a Z(zw TzT...4v Zai)? 
NN N d 


Expanding the square, this expression becomes 


2z}; Dak; 22; Ezra; Z2, Zoi 

T, = s mds tee Pa te CENT deese AR T 
' 

Now, since 


(2.39) 
the last formula reduces to 


2 
j<k=1 


Similarly the standard deviation of the variable w may be written 


9 
(2.40’) ow = |Q + » 2, fx. 
Ske 
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Returning to formula (2.38), it is seen that there remains Sv;w;/N to be 
expressed in terms of correlations between the original variables. Inserting 
the values of v; and w; into this formula, there results 


N a Q 
ydus = Y EX) : 


If the sums in the parentheses are expanded and the indicated multiplica- 
tion performed, this expression can be rearranged in the following form: 


N a Q 
3 ( > 2 
Zvw; 


i21 \j=l1 J=I 


N N i 


which, in turn, upon summing for 7 reduces to 


Q 
Zvw; z 
ean ne S Sry. 
j=1 J=I 


Substituting the values from (2.40), (2.40’), and (2.41) into (2.38), the fol- 


lowing expression for the correlation between two sums of variables is ob- 
tained: 


a Q 
> ye 
_ j=l J-I 


Tow = SERE 
9v0w 


(2.42) 


This formula gives the correlation between any two composite variables in 
terms of the correlations between the individual variables comprising the 
derived ones. 

As a special case of (2.42), the correlation of a variable Zo in standard 


form, with a composite variable v may be considered. It is readily seen that 
the correlation in this case is given by 


a 


Èr 
(2.43) PEE 


To = — 


EM 


i<k=1 
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It may be instructive to give a diagrammatic representation of formula 
(2.42). Consider the following table of intercorrelations between the original 
variables involved in v and w: 


Il Q 


Tat 


Tin 


Let Aq represent the sum of the elements in the upper triangle, that is, 
the sum of the intercorrelations of the q variables making up the compos- 
ite variable v. Similarly, let AQ represent the sum of the elements in the 
lower triangle; and C1qQ those in the rectangle, that is, the sum of the in- 
tercorrelations of the variables involved in v with those in w. Then formula 
(2.42) may be written symbolically in the form 

oQ 


dn he = oF Gg VO F SAU 


2.11. Application of Correlations of Sums 

To indicate the use of the formulas just developed, they will be applied 
to the groups of tests defined in (2.35). The composite variables arising 
from these groups are 


9 


(2.45) actin = > t a= > 4%) v= > ts n = Š ži, 


j=1 j=5 j=10 j-214 j=20 


where a single letter v with varying subscripts is employed to distinguish 
the variables. 
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The standard deviations of these new variables may be computed by 
means of (2.40), the result being 


c, = V4 + 22.041) = V 8.082 = 2.843, 


T», = V5 + 2(6.378) = V17.756 = 4214, 
(2.46) os, = V4 + 2(2.952) = V 9.904 = 3.147, 
v, = V6 + 2(4.782) = 15.564 = 3.945 , 
oo, = V5 + 2(4.336) = V13.672 = 3.698 . 


TABLE 2.4 


INTERCORRELATIONS OF FIVE 
COMPOSITE TESTS 


Composite 
Test 


T3 u vs 


.495 | .497 | .460 |. 
.594 | .649 | :545 


TABLE 2.5 


CORRELATIONS OF ORIGINAL WITH 
COMPOSITE TESTS 
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Then, applying formula (2.42), the intercorrelations of the composite tests 
are readily obtained. They are presented in Table 2.4. 

Besides the intercorrelations of composite tests, there will be occasion to 
use the correlations of the original twenty-four tests with the composite 
ones. Such correlations may be computed by means of formula (2.43). In 
Table 2.5 are given the correlations of each test z; (j = 1,2, . . . , 24) with 
the composite tests v, (s — 1, 2, 3, 4, 5) which do not include the test z;. 


CHAPTER III 
GEOMETRIC FORMULATION OF THE FACTOR PROBLEM 
3.1. Introduction 


For a complete understanding of the various factor methods for analyzing 
a set of variables it is very helpful to resort to geometry to supplement and 
extend the algebraic ideas. The geometric foundation to be developed in 
this chapter will furnish a basis for subsequent analysis and comparison of 
methods. It will then be possible to use geometric ideas freely and also to 
pass from these to equivalent algebraic expressions. The geometry which is 
most useful for factor analysis is that of higher dimensions. The concept of 
extending the idea of dimensionality has been confined to the comparatively 
small class of mathematicians and philosophers, but psychologists and edu- 
cators have now come to rely on this concept as a very useful tool in analyz- 


ing mental traits, the total number of dimensions required being certainly 
greater than three. 


After a very brief exposition of 
try, a coordinate system is introduced 
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3.2. Geometry of N Dimensions 


The concept of higher dimensions is arrived at by geometric and alge- 
braic means. The notions of point, line, and plane may be generalized to 
higher dimensional objects, and extended geometric interpretations of al- 
gebraic relationships may be given. The deductions made in the higher 
dimensional spaces are based upon the analogous theory in three-dimension- 
al space and supplementary logical reasoning. 

The basic axioms* for Euclidean geometry may be assumed and such 
modifications made as are necessary to insure that the space has a sufficient- 
ly high dimensionality. The point, straight line, and plane are taken as 
undefined elements, and later corresponding elements of higher dimensional 
space may be defined in terms of these. 

Starting with four given non-coplanar points, all the points, lines, and 
planes can be obtained which constitute a three-dimensional space. The 
space, or manifold, determined by these points is essentially ordinary space 
of three dimensions. If the existence of at least one point not in this set is 
postulated, then all the elements of a four-dimensional space can be ob- 
tained (with the help of the basic axioms). The three-dimensional region 
does not now constitute the whole of space but merely a subspace of the 
space of four dimensions. The three-dimensional region will be called a 
hyperplane lying in the four-space, analogous to a plane lying in a three- 
space. A hyperplane in a space of four dimensions is determined by four 
non-coplanar points, a point and a plane, or by two skew lines. 

Some of the elementary geometrical properties of the elements in a three- 
and four-dimensional projective space may now be enumerated.} In a three- 
dimensional space two planes intersect in a line; à line cuts a plane in a 
point; and any three planes have a point in common. Ina four-dimensional 
space two hyperplanes intersect in a plane, three hyperplanes intersect ina 
line, and four in a point, while five do not in general have any point in com- 
mon; a hyperplane cuts a plane in a line, and a line in a point; two planes 
have in general only one point in common, and a plane and a line in general 
have no point in common. 

The notion of dimensionality may be viewed in another manner. À point 
in a line is said to have one degree of freedom (of motion); in a plane, two; 
and in ordinary space, three. The point being taken as element, à line is said 
to be of one dimension; a plane, two; and ordinary space, three. These 

* D. M. Y. Sommerville, An Introduction to the Geometry of N Dimensions, 1929, 


Chap. I. See Appen. B.4 for list of basic axioms. 
t By postulating & projective space the statement of the geometric properties is greatly 
simplified by not having to deal with special parallel instances. See Appen. B.4 for the 


projective axiom. 
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spaces are called linear spaces, or flat spaces, i.e., a plane is a two-flat and 
ordinary space is a three-flat. An (N — 1)-flat in an N-space will generally 
be called a hyperplane. The linear spaces point, line, plane, three-flat, . . 
hyperplane, N-flat are manifolds determined by one, two, three, four, . . 
N, N +1 points,* respectively, and having zero, one, two, three, .. 
N — 1, N dimensions. 


EET 
ay 


og 


3.3. Cartesian Coordinate System 


The geometric ideas are found to be most useful and easily formulated 
when they are given analytic representation. A point P may thus be repre- 
sented by a vector (zi 22,..., zw), which is a matrix with one row (or 
column). Each z; is a real number, and all N numbers may be called a sys- 
tem or N-tuple. By a “point” is meant simply one of the undefined elements 
of the space which is characterized by a given set of axioms, so that a set 
of points is really an arbitrary set of any whatever elements. On the other 
hand, the N-tuple (21, 25, . . . , zy) may be called an “arithmetic point." A 
correspondence between a set of “geometric points" and a set of “arithmetic 
points" is called a coordinate system.t The numbers, zi, %2,..., zy, which 
constitute the representation of P, are called the cóordinates of P. For 
purposes of factor analysis, the distinction between a "geometric" and the 
corresponding "arithmetic" point is not essential, and the word “point” will 
be used for either one. The notation P : (x,) will frequently be used to 
designate the point and its coordinates. 

An N-dimensional Euclidean space will be assumed, an 
non-homogeneous Cartesian coordinate system will be set up. The points 
O : (0, 0,....,0), and E25 (1, Os 


r :,0), Ez: (0, d Oy nce gO), m c 
Ey : (0, +++ 0, 1) will be called the origin and unit points, respectively. 
The N lines Oz; (i = 1, 2,..., N), each passing through the origin and 


one of the unit points, will be called the coordinate axes. The N hyper- 


planes m; = Oxia, se )ai(... ty, each passing through O and containing 
N — 1 axes, will be called the coordinate hyperplanes. A hyperplane 7; is 


d in this space a 


* It is understood that the set of P points, which determine a (p — 1)-flat, do not lie 
in à (p — 2)-flat. " , 


T The coordinate systems that will be introduced in this vol 
a one-to-one correspondence between thi 


£ ; OF an even more gene: 
may prove to be very important, but thus far such research has 


not been undertaken. 
1 The inverted parentheses are used in the designation of an h; indi 
] y erplane t. 
the omitted coordinate axis. — ine 
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said to be “opposite” to the axis Oz;. The coordinates (zi, 2s, . . . , £y) of 
any point P are equal, respectively, to its distances from each coordinate 
hyperplane measured along a line parallel to the opposite axis; or, in other 
words, the distance cut off on each axis by a hyperplane parallel to the re- 
spective opposite coordinate hyperplane. For exemple, the coordinate 2; is 
equal to the distance (denoted by xı) cut off on the Oz; axis by a hyperplane 
parallel to the coordinate hyperplane rı. 


3.4. Linear Combination and Dependence 

The N-tuple (a, 2s, . . . , ty), which represents a point P, may be con- 
sidered as a vector which joins the origin O to the point P. Such a vector is 
sometimes called a “radius vector." Two fundamental operations in vector 
algebra are multiplication by a number and addition of vectors. More pre- 
cisely, if P is a point represented by the vector (zi, tə . . . , zy) and c isa 
number, then cP is the point 


(cz, C29, ..., Cty), 


while, if Py : (zu, tw, - . . , Ziv) and P3 : (Ta 225, . . . , Zav) are two points," 
then P, + P; is the point 


(zu + Ta, tro + Xm, ll Tin + Ton) . 


In general, any linear combination of m points, Pi: (zu, 2: . .. , Zw); 
2, Pm! (Eon Zm2) +++ s tm), may be defined by combining the two pre- 
viaus operations, as follows: 


LP) + GPs + o.c Ps, 


where the /'s are any numbers. By taking varying values of the ?'s, different 
linear combinations of the original m points can be obtained. Any one of 
these new points may be denoted} P(t) or P(t, t», . - . , tm), with coordinates 


given by 


m 
(3.1) som > bet = 1,30. 0M: 


* The double subscript notation is used on the coordinates in order to distinguish the 
points. Thus z,; designates the ith coordinate of the point P,. 

t The symbol P(t), or P(t, 4, - . +, tm), is the conventional function notation which 
is to be read, “P is a function of ¢ (in this case, a set of t'), or Pis a function of th, ta 
. «+4 tm’? On the other hand, P : (z;) is the notation for a point P with coordinates zs. 
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and are said to be linearly dependent on the original points Pi, Ps. vey Pss 
Each coordinate z; of a point P(t) is expressed as a linear combination of 
the corresponding coordinates zi; Toi, . - - , Tmi Of the m points Pj, Ps, ..., 


P,,.* A set of two or more points is said to be linearly independent if no 
one of them is dependent on the rest. According to these definitions, the 


origin is linearly dependent on any set of points and a single point is linearly 
independent if it is not the origin. o 
The preceding definitions can be made, alternatively, by giving a direct 


definition of linear independence. Thus, a set of points P1, . . . , Pm is linear- 
ly independent if the conditions 

m 
(3.2) S is c 0 

a=1 


imply that 4 = t=... = t, = 0. This is readily seen to be consistent 
with the definition (3.1). For if one of the coefficients were different from 
zero, say tı ~ 0, then (3.2) could be written in the form 


ty ty t 


m 5 " 
de — pude c rap — $ Tmi; 


i, i, i D 


and, according to (3.1), the point P, would be one of the points P(t) which 
is linearly dependent on the points Po, P3,..., Pm. Having a positive 
definition of independence, the definition of linear dependence is given by its 
negation, that is, a set of m points is linearly dependent if the conditions 
(3.2) hold for the coefficients not all zero. 

When a set of points is given, it may be of interest to know how 
them are linearly independent. Let Pi 
oo ap Manly soe Put aya, ws 


many of 
> (Gu, muy... mw), Pe: (t21, X22, 
* » Zay) be any set of n points. Either all 


* To clarify some of these ideas, consider the special case of N 


Py : (au, 212, 213) and P; : (zn, Zo, 225). All the points P(t) 
on the points P; and P; are given by the follow: 


— 3 and two points 
which are linearly dependent 
ing coordinates: 
Ti = brn + bra 
P(h, t) : 4 zy = liz + btn 
T = liT + rs 
for varying values of 4, and b. For 
P(t) is a linear combination of the 
is the same linear combination of t 


, if the coordinates of P, are (1, 3, 
= 1, t = 2, then P(t) is given by the coordinates 
a = 5,22 = 5,23 = 14. 
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these points coincide with the origin or at least one of them, say Pi, is in- 
dependent. Of the remaining points, either they will all depend upon P; or 
at least one of them, say Ps, will be independent of P,. Proceeding in this 
way an independent set of points, say Pi, Ps, . . . , Pm, will be obtained upon 
which all the points Pi, Ps, . . . , Pa will be linearly dependent. A criterion 
for determining m may be obtained by means of the matrix 


|zu te Tis... Ty 
| 
| Ta Ta Tag ... Ton 
leill = || Ea zs s .-. Tax jjo 
Tni Tne Tn3 +++ Tan 


whose rows are the n points. An important result for linear dependence of 
points, and which will be utilized later to determine the number of common 
factors necessary to describe a set of variables, may be stated as 
Tueorem 3.1. If m is the rank of the matrix lz;ill, the points Py, Po, ..., 
P, are all dependent upon m of them, which are themselves independent.* 
Subspaces of the N-space may now be given analytical representation. 
If Pi, Py, ... , Px are k linearly independent points, the set of all points 
linearly dependent on them is called a linear k-space and is defined by the 


equations 


k 

(3.3) i= > tits (um 1,8, casu N] 
j=l 

where the ¢’s are a set of k parameters, and for each set of values (ti; 5, .. . s 


tx) there is a corresponding point of the linear k-space. Any one of the origi- 
nal k linearly independent points is, of course, given by definition (3.3); for 
example, P; is given by ti = 1 and & — i — ... = tą = 0. The k points 
Py, Ps,..., Px are said to determine the linear k-space. A linear 1-space 


* For a proof of this theorem see Appen. B.5. An application may be made to the 
example of the last footnote. The matrix of the three points is 


13 4 
2 1 85]; 
5. 5 14 


which is readily seen to be of rank two; for the third-order determinant is zero while a 
second-order determinant (e.g., the one in the upper left-hand corner) can be found which 
is different from zero. Since the matrix is of rank two, the three points are dependent 
upon two of them, which are themselves independent. Then the three points are con- 
tained in a plane, but not in a line, as will be evident from Theorem 3.2. 
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consists of the points whose coordinates are proportional to those of a given 
point Pi : (xn, 2:5, . - . , iw), and may be called a line through the origin. 
Its equations are given by 


(3.4) ti = mu Gal 2... N). 


These are a set of N parametric equations of a line through the origin, where 
tı is known as the parameter. 

The transitive law for linear dependence may now be indicated. All 
points linearly dependent on m points Pi, .. . , Pm in a linear k-space are 
contained in that k-space. The coordinates of the m points are given by 
equations of the form (3.3), and any point linearly dependent on P, . . 
P, is then obviously dependent on Pi, . . . Px 

Furthermore, if the points P;,..., Pa determine a linear k-space, there 
is no other linear k-space containing these points. A linear k-space is thus 
determined by any set of k independent points contained in it, and a linear 
k-space does not contain a set of l independent points, where | > k. For, by 
definition (3.1), it is implied that any k points in a set of l independent 
points are themselves independent, and hence determine a linear k-space 
containing the larger set. Theorem 3.1 may then be stated as follows: 

THEOREM 3.2. If m is the rank of the matriz \|x;:l|, the points Py, Po, . . . 5 


P, are all contained in a linear m-space but not in a linear u-space, where 
uL «m. 


oF 


A geometric interpretation of linear dependence can now be given. The 
m vectors P, : (Tai, Toz, ..., zw), (S = 1,2,... , m), employed in the defi- 
nition (3.1), determine an m-dimensional subspace of the original N-space, 


and if OPs, OP», .. . , OP, are taken as the coordinate axes, then fi, i» 
- , min (3.1) are the coordinates of z;. 


A linear k-space, as defined by equations (3.3), 
since the origin is linearly dependent on any se 
subspaces of the N-dimensional Space may be 
do not include the origin. For this purpose, 


(3.5) 


always contains the origin, 
t of points. The notion of 
generalized to spaces which 
a translation of coordinates, 


yi = zie, 


is defined. Then any set of points which corresponds, under a translation, 
to a linear k-space may be called a flat k-space, or merely a k-flat. As noted 
in 3.2, a 0-flat is a single point; a 1-flat is a straight line; a 2-flat is a plane; 
and an (N — 1)-flat is a hyperplane. 

The geometric theorems developed in this section will be applied specifi- 
cally to the factor problem in 3.8. First, certain fundamental formulas for 
distance and angle will be developed and applied to correlational theory. 
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3.5. Elementary Formulas in Rectangular Coordinates 

When the coordinate axes are mutually orthogonal, i.e., at right angles 
to one another, the reference system set up in section 3.3 is called a rectangular 
Cartesian system. Some elementary formulas in rectangular coordinates will 
be presented in this section. 

For any two vectors or points P : (tu, Ti», . . . , tiv) and Ps : (Za, To, 

. , zay), their scalar product* is defined by 


(3.6) Py + Po = Etita, 


where summation with respect to 7 is understood. Then the norm of P; is 
defined as the positive square root of the inner product of P, with itself, 
that is, i 


(3.7) N(P) = VP, + Pi = VEx, 


and the distance between P; and P» is defined by 
(3.8) D(P:P2) = N(Pi — P?) = VZ(n; — 23). 


It is readily seen that the norm of a point is the distance from the origin 
to the point, that is, N(P) = D(OP). The distance function satisfies the 
following familiar conditions of elementary geometry: 


D(hYP) = 0, 

D(PiP3) # O if Pi # Ps, 
D(P:P2) = D(P2P:) , 

D(P:P2) + D(PP3) = D(PiPs) - 


(3.9) 


The first three of these relations are obvious. The fourth, however, re- 
quires some proof. It may be noted that distances are invariant under 
translations. Thus if two points Pı, P; are translated into two points Pj, 
P}, then D(P1P;) = D(PIP2), which may be verified by applying (3.5). 
The fourth formula of (3.9) will therefore be unaltered if the points Pi, Ps, 
and P; are transformed by a translation which carries P» into the origin. 
Then, putting in the distances by means of (3.7) and (3.8), the inequality of 
(3.9) becomes 


(3.10) VZEx + Vith m VXQns— vs, 


which may be verified algebraically. 


* This is sometimes called the inner or dot product. 
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Now the equality occurs in (3.10) if, and only if, 
Tzi = — lT t= 1; 2,...,N), 


where £j is a positive constant. These equations are of the form (3.4) and 
so represent a straight line through the origin with the points P, and P, on 
opposite sides of the origin. Hence, equality occurs in the fourth relation 
of (3.9) if, and only if, the coordinates of Py, P», and P; are related by equa- 
tions of the form 


(3.11) A(xii — zai) + B(zs; — za) = 0 , 


where A and B are constants of like sign and not both zero. If the condi- 
tion (3.11) is satisfied, and if P, z P; and Ps z Ps, then Ps is said to lie 
between P, and P3. 

Of special interest to factor analysis are some theorems of ele 
geometry which have to do with transformations which 
invariant. Such transformations, in which any point P, 
into Qi : (yi) and P» : (2s;) is carried into Q; 
property 


mentary 
leave distances 
: (ru) is carried 
: (2), have the following 


(3.12) E(ni—m)- E(yi— ys). 


From the condition that a point P; is between two others, P, and P, if, 
and only if, 
D(P,P) + D(P2P3) = D(P,P3), 


it follows that a transformation which 
straight lines into Straight lines. Now, 
ometry,* the transformation is linear, t 


leaves distances unaltered carries 
by a fundamental theorem of ge- 
hat is, of the form 
N 
(3.13) Yi = Dd anta T ci 
k=1 
Q-12.. Myf «1, BB cad 
Upon substituting the values of Wi 


Die and ys; from (3.13), equation (3.12) be- 
N N N " 

(3.14) PX — Tx)? = >, [S euen = o] è 
i= t=1 k=1 


* 

" bri ak. states that any nonsingular transformation of an N-space into itself is 

oe if ù "rn lines into straight lines. For a proof of this theorem see Oswald 
eblen and J. H. C. Whiteh p i 3 7 

me itehead, The Foundations of Differential Geometry, 1932, pp. 


GEOMETRIC FORMULATION OF FACTOR PROBLEM 49 


It now remains to find the conditions which the a's must satisfy in order 
that (3.14) should hold, and then the most general transformation which 
preserves distance will be specified. The right-hand side of (3.14) can be 
written as follows: 

N 
> [ef (tu — ta)? + afa (rio — ta)? +... + ains — tay)? 
i=l 
+ 2[eanais(zu — Ta) (t — 223) +... 
Taj scaaiv(ri y- — Te vi) (iv — Twy)]] D 


or simplified to 


N N N 
P, >, M ainain(are = Tor) (£ü — za). 


Hence, equation (3.14) is satisfied when 
(3.15) Xaian = Ôr, 


where ô+ is the Kronecker delta which is equal to unity if k = l and equal 
to zero if k = l. Any linear homogeneous transformation, 


N 
(3.16) Yji = M astm 
k=1 


whose coefficients satisfy (3.15) is called orthogonal, and its matrix an 
orthogonal matrix. The following theorem has thus been established: 

THEOREM 3.3. The distance between any two points ts an invariant under 
@ general rigid motion, that is, an orthogonal transformation followed by a 
translation. 

Other geometric ideas that are useful in factor analysis center around 
the notion of the angle between two lines. The only characteristic of a point 
is its position, as given by its coordinates in a frame of reference. A line is 
ordinarily distinguished not by coordinates but by its inclinations to the 
respective coordinate axes. The angles which a line OP makes with the 
axes, i.e., 0; = POzx,, are called the direction angles of the line, and their 
cosines are called direction cosines. If the norm N(P), i.e., the distance 
D(OP), is denoted by p, then 


(3.17) zi = pcos 6; (i 


By (3.7), 


P = g, 
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and substituting the value of x; from (3.17), gives 


p = Ep cos? 6; , 
so that 


(3.18) Zz cos? 6; = 1. 


This property, that the sum of the squares of the direction cosines of a line 
in N-space is equal to unity, is a direct extension of the one in ordinary space. 

The parametric equations of a line through the origin O and a fixed 
point P; : (x;) are given by (3.4). The coordinates of any point P : (2;) 
on a line through the origin with the direction cosines 


^; = cos 6; 
are given by (3.17). When pis taken asa parametric variable along the line, 


the N equations (3.17) can be regarded as the equations of the line, which 
may be written 


Ti > 
im Ged, n Ms 
Upon equating the N expressions for p, the following N — 1 equations 
arise: 
(3.19) ee, ee ae 


IP. ta 2.25 eu) Je taken as a variable point on the line, (3.19) 
can be regarded as the equations of the line. 

By means of a translation, of the form (3.5), the equations of a line AP 
through an arbitrarily fixed point A : (a, a,..., ay) and with the direc- 
tion cosines ^; are transformed from (3.19) to 


(3.20) Tı — a = X9 — de = EX XN — ay 

we om S ER ed 
Moreover, if 
(3.21) X = bl; (62 1,2, ND, 
where b is a constant different, from zero, the equations of the line AP may 
be written in the form 
(3.22) $— z-—a — tN — an 

i Tg Sers A, 


ly 


———————————À  —— "MI 
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where the /; are not now equal to, but only proportional to, the direction 
cosines. The numbers l; are called direction numbers of the line. 

The actual direction cosines of a line can readily be obtained from the 
numbers proportional to them. For, squaring both sides of (3.21) and sum- 
ming for 7, this equation becomes 


PIR = == 1, 


where the last equality follows from (3.18). Then the constant of propor- 
tionality is 
b= 1 
VER’ 


1 
and the direction cosines are given by 


l: 


Hence equations (3.22) may be taken as the general form of the (N — 1) 
equations of a line in N-space. risus 
The coordinates of any point P : (xj) on a line through A : (a) with di- 


rection numbers l; are 
(3.24) ny = ast Ui @=1,2,....™), 


where ¢ is the repeated value in (3.22). Equations (3.24) may be regarded 
as a system of parametric equations of a line through a fixed point. The 
distance D(AP) along the line from the fixed point A up to any position of 
the variable point P is 


D(AP) = VE: — a? = tV Zl, 


so that, 
D(AP) 
(3.25 = $ 
) t E 


It is thus evident that the parameter ¢ in equations (3.24) is proportional 
to the distance from the fixed point to a variable point on the line and is 
equal to this distance when the equations of the line are given in terms of 


the direction cosines. E 
Now a formula for the cosine of the angle between two lines in N-space 


may be derived. When two lines meet in à point,* a plane can be drawn 

* If the lines do not meet in a point, the angle between the lines may be defined as the 
angle which one of the lines makes with a line parallel to the second, which intersects 
the first line. 
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through the point containing the two lines, and their inclination can be 
obtained from the trigonometric properties of a triangle in the plane. Let 
the two lines through A : (a,) be represented by the equations 


ty — Gy — T= ay Ty — ay 
x " TT Nw s 
Y= 1 _ Y-a _ z Ue — Gar 
T i — nr s 


(3.26) | 


where the z; and y; are the coordinates of the variable points on the lines, 
and the ^; and y; are the direction cosines of the lines. On the first line take 
any point P at a distance p from A; on the second line take any point Q 
at a distance q from A; and connect the points P and Q with a line, which 


necessarily lies in the plane. The points and lines are plotted in the plane 
of the two given lines in Figure 3.1. 


A:(a;) q Quy) 
Fic. 3.1 
Let $ = angle PAQ and let d = D(PQ), then the law of cosines applied 
to the triangle PAQ gives 
(3.27) 


Ë = pP + @ — 2pq cos 4. 


The distance d is also given b 


the coordinates of P are z; 
so that 


y formula (3.8), in which, aecording to (3.24), 
= a; + p^; and those of Q are y; = à; + qui 

d* 
(3.28) = 


R(t: — yi)? = E(pM — qui) , 
PEN + PER — 2pqXXg j 
P te — 2pgZXa;, 

since ZN = Tu? = 1 by (3.18). W 
with the corresponding ones of (3.2 


(3.29) 


hen the terms of (3.28) are identified 
7), the following result is obtained: 


cos ó = Za. 
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Thus the cosine of the angle of separation of two lines is given by the sum 
of the products of corresponding direction cosines of the lines, i.e., the scalar 


product of the vectors (Mi, Xs - . . , Aw) and (11, Ma, . . +» ux) 

If the equations of the lines are given in the form (3.22), with l; and m; as 
the direction numbers of the respective lines, the cosine of the angle of 
separation becomes 
Ilim; 


ZB Vm 


(3.30) cos ó = 


Here the cosine of the angle of separation of two lines is given in terms of 
numbers proportional to the direction cosines of the lines. 

By means of formula (3.29) another expression for the scalar product of 
two vectors or points may be written iz: place of (3.6). The coordinates of 
the two points Pi : (i) and P» : (15) may be expressed as follows: 


Tii pii, Tor = prs GS, 2, eeN); 


where pi, p» are the respective distances from the origin O to the points 
Py, P», and Xi; Xs are the direction cosines of the lines OP; and OP». Then, 


substituting these values in (3.6), there arises 


Py + Po = Xn ppX, 


which, according to (3.29), reduces to 
(3.31) P, + Po = pip: COS diz, 


where $i; is the angle P,0P;. Formula (3.31) states that the scalar product 
of two vectors is the product of the lengths of the vectors by the cosine of 
their angular separation. This is very often taken as the definition of the 


scalar product. 


3.6. Elementary Formulas in General Cartesian Coordinates 


In the preceding section various formulas were presented in terms of 


rectangular coordinates. Now the restriction that the coordinate axes are 
mutually orthogonal will be removed, and more general formulas obtained. 
The formulas for distance and angle will then be in terms of general Car- 
tesian coordinates and will simplify to those of 3.5 when the angles between 
all pairs of reference axes are taken as 90°. 

The general Cartesian coordinate system contains N reference axes Ox; 
which may make any angles with one another. Then the angle between the 


ve 
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xz; and x, axes may be designated 0; (i, k = 1, 2, ... , N). As may be 
expected, the formulas for distance and angle in terms of general Cartesian 
coordinates will involve the inclinations of the reference axes. 

Formulas for the distance function in general coordinates will first be 


given. In the plane the square of the length of the radius vector OP is 
readily found to be 


p? = [D(OP)f = 23 + 23 — 2xirs cos (180° — 05) 
= 2} + T} + 2x2 cos O12. 


X; 


M:(x,, 0) 
Fia. 3.2 


This formula follows immediately on applying the law of cosines to the 
triangle POM, indicated in Figure 3.2. By induction, it can be shown that 


in N-space the distance p from the origin O to an arbitrary point P : (ax 
7o, ... , tx) is given by 


(3.32) p = V ZXz;t, cos 0; , 


Lo ZZ indicates summation for i and k from 1 to N. This convention 
or the double summation will be employed throughout this section. In a 


ve cibi ger o the distance between any two points P, : (£u, zi; , 
Tın) an © @ i i lowin 
e 2: (Yn, 25, . . . , Tey) may be shown to be given by the following 


(3.33) 


D(P;P2) = VEE (a; = oi) (Tix = Tok) cos Bi 
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'The relations (3.32) and (3.33) reduce to the corresponding formulas (3.7) 
and (8.8) for distance in rectangular coordinates when the angles of inclina- 
tion between pairs of axes are 90°, inasmuch as cos 6;, = 0 for every i ¥ k. 

There will now be given several properties of a line in terms of more 
general (not necessarily rectangular) coordinates. The direction of a line 
OP is determined by the ratios of the coordinates of an arbitrary point 
P : (xy, 25... , zy) on the line to the length p = D(OP). These ratios, 
denoted by ^; = z;/p, are called the direction ratios of the line OP. Then if 
the coordinates of P are expressed in the form 


(3.34) Ti = pM or Tk = pry, 
and substituted in (3.32), there results 


p = pZZNO cos Oik , 
so that 


(3.35) EEN: A cos O54 = Ls 


The direction ratios become the direction cosines of a line when a general 
Cartesian coordinate system is specialized to a rectangular one. Then for- 
mulas (3.34) and (3.35) reduce to (3.17) and (3.18), respectively. 

An expression for the angle between two lines, in general coordinates, 
can now be deduced. For simplicity, let the two lines pass through the ori- 
gin and be distinguished by the direction ratios A; u; (? = 1, 2,..., N), 
respectively. Select a point P : (z;) on the first line, and a point Q : (yj) 
on the second line, and let p = D(OP), q = D(0Q), d = D(PQ), and ¢ = 
angle POQ. Then 

d = pP? + gẹ — 2pg cos à. 


But d is also given by (3.33), in which the coordinates of P are z; = ph; 
and those of Q are y; = gui, so that after these values are substituted, the 
formula becomes 


d?— p dg -—2pgZZNu; COS bix . 


By equating the two expressions for the square of the distance, the following 
formula for the angle between two lines is obtained: 


(3.36) cos ó = ZZ COS Giz - 


This formula reduces to (3.29) when the axes make right angles with one 
another. 
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3.7. Geometric Interpretation of Correlation 


In this and the following section there will be presented a number of ap- 
plications of the preceding geometric ideas to the factor problem. The raw 
data are the values of n statistical variables for each of N individuals. These 
data may be set down in a matrix as follows: 


Xn Xe Ms Xix | 
Xa Xe Xa Xas |l 
iem = | Xs Xm Xa o... 2 (ay , 
EX. Zu Xa €] 


where any element X ;; represents the observed value of variable j for indi- 
vidual 7; so that the n rows represent the variables and the N columns, the 
individuals. By subtracting the mean of each variable from the observed 
values of that variable, the matrix of deviates, 


| £u 


Xi) Tiz Ti | 
Toi Ta T23 Tow | 
lezl] = | za xa Tas Tay | " 
v | 
Tri Inq Tng Tan | 


is obtained. 

Each row of ||z;:|| consists of N real, ordered numbers, and hence may 
be considered as the rectangular Cartesian coordinates of a point x; (j = 
1, 2, ---, n). The variables (or points) are thus designated by an H with 
a single subscript, whereas the values of the variables (i.e., the coordinates 
of the points) are indieated by an z with two subscripts. puse cem value 
(or coordinate) z;; stands for the ith coordinate of the point x 

As has been remarked before, it is usually more Poker: d to put the 


variables in standard form The matrix of sta i ed v. en 
. z tandardiz 
alues may ther 


| 21 Z 


213 Zin 
| 
Zn Zo  Zos Zan 
T 
laid] = | 231 Za Zaz Zan || , 
E 
[Zai Zn? Zn3 Zan 


in which the rows may be inter 
preted as the rectangular G i ^ i 
nates of n points z; (j = 1, 2, . : eee mene 


. n) in an N-space. 
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The length of the radius vector to a point z;, according to formula (3.7), is 
(3.37) py = VE. 


According to equation (2.1), however, the expression on the right is equal 
to V/No;, so that 


(3.38) p; = VNo;. 


The standard deviation of a statistical variable may thus be interpreted as 
being proportional to the distance from the origin out to the point represent- 
ing the variable, the constant of proportionality being 1/VN. 

By way of geometric representation of a set of values of two variables, 
x; and z;, it is customary to think of the z;; and the zx: as the coordinates 
of N points (zi, ti), (ty, Tr), .. . , (Zin, Tex) in the plane zjOz,. This 
plot of points is called a scatter diagram; and, by means of this representa- 
tion, a better understanding of the relations involved in the definition (2.3) 
of a coefficient of correlation can be obtained. In general, for n variables 
this will be referred to as the point representation. 

Even more important in some respects is a geometric representation not 
by N points in a plane (for two variables) but by two points in an N-space. 
The two variables are then represented by the points x; : (zi, Tje -> Xin) 
and z, : (tm, 242, ..., Zev), Which may be termed “vectors” according to 
3.4. Such a configuration for n variables will be called the vector representa- 
tion. 

If the direction cosines of these vectors are denoted by Aj; and Axi, re- 
spectively, then by (3.17), 


(3.39) =H, dee SP = 1,2,...,9), 
Pi m 


where p; = D(Oz; and p, = D(Oz;). Inserting these values in formula 
(3.29), it becomes 
Zzjuki 

i MAT IRI 
(3.40) COS jr = EN. non. 
where ¢;, is the angle of separation of the two lines. Then cos $;, may be 
interpreted as the scalar product of the vectors z; and zi divided by the 
product of the lengths of these vectors. Upon substituting the values for 
^; and p; from (3.38), formula (3.40) reduces to 


Zrütu 
Nojox 


(3.41) cos jk = 


uL 
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which, according to (2.3^), becomes 
(3.42) Tik = COS Qik G 51,2... m). 


The coefficient of correlation between two variables (measured as deviates from 
their respective means) is the cosine of the angle between their vectors in. N-space. 


3.8. Fundamental Subspace for Factor Analysis 


By means of the geometric notions introduced in this chapter it is pos- 
sible to determine the minimum number of common factors that is neces- 
sary to describe a set of variables in the sense of equations (2.16). Accord- 
ing to Theorem 3.2, the n points whose coordinates are given in the matrix 
|[z;;|| are all contained in a linear m-space, where m is the rank of the matrix. 
In other words, the n vectors can be described in terms of m reference vec- 
tors. 

This property can be stated in terms of the intercorrelations of the n 
variables by means of the following: 

THEOREM 3.4. The rank of the product of 
to the rank of the matriz.* 

The product of the matrix l|z;:|| by its transpose, llz 


a matriz by its transpose is equal 


ill, is given by 


Z2 Za o ZXAga o... Z2ysi 

Zzzni Lz}; Z2yizsi Z2oiZ5i 
(3.43) llzssd + Meal = || Ezazu Zzxzo; Dz}; Zzsini 

Ze Zza BZpiZai 222; 


Recalling that 22}; = N and Zzjz4 = Nrj, the preceding equation can be 
written in the form 


l Tg fn eak Tin 
v T 1 Toa Ton 

(3.44) |l e zall =N rm ra 1 Tan | = N [ri . 
Tui Tre Tag ... 1 


Then, according to Theorem 3.4, th 
equal to the rank of the matrix of 
property of the variables which is 


e rank of the correlation matrix ||7;,|| is 
f standardized values, llz;i]|. Hence any 
inferred only from the rank of the latter 
* For a proof of this theorem see Appen. B.6. 


O— KP RP  o 
——- MAT w————————HQud——PÉ——— 
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matrix may be stated in terms of the correlation matrix. It therefore follows 
from Theorem 3.2 that the n variables can be expressed as linear functions of 
not less than m factors, where m is the rank of the correlation matriz. 

In case the correlation matrix contains ones in the diagonal, its rank is n, 
and the variables would then be describable in terms of n common fac- 
tors. If it is desired to describe the n variables in terms of less than n com- 
mon factors, a pattern of the form (2.16) may be postulated. From such a 
pattern the correlations are reproduced, as before, but with communalities 
in place of ones in the diagonals. Then a factor pattern of the desired form 
can be obtained by employing a matrix of the form in (3.44) with the ones 
replaced by communalities. The rank of this matrix is generally less than 
the order n. By the preceding argument, it is therefore apparent that the 
number of common factors in the pattern is equal to m, the rank of the cor- 
relation matrix. This is the smallest number of factors that will account 
for the intercorrelations. Stated geometrically, the smallest space contain- 
ing the n points is a flat m-space. Such a space will be referred to as the 
common-factor space. For purposes of reference the above ideas may be re- 
capitulated in the following: 

THEOREM 3.5. If m is the rank of the correlation matriz, with communalities 
in the diagonal, then the smallest number of linearly independent factors which 
will account for the intercorrelations is m; or, the common-factor space is of 
m dimensions. * i 

In order to clarify the preceding ideas, the three important spaces will 
be reviewed. For any variable z; the system (2j, is . . . , Ziv) of N real 
numbers may be considered as the rectangular Cartesian coordinates of a 
point in an N-dimensional space. By means of this vector representation, 
the configuration of two variables is merely two dimensional, i.e., in a plane, 
although it has to be regarded as imbedded in an N-space. In general, the 
configuration of n vectors may be regarded as in an n-dimensional space 
Which is imbedded in the original N-space. For purposes of factor analysis, 
this space can be greatly reduced, as indicated in Theorem 3.5. 

Before giving the final interpretation of the vectors representing the 
variables in the common-factor space, a geometric discussion of the linear 
expressions (2.4) which include unique as well as the common factors will 
be found convenient: The n vectors may then be considered in the total- 
factor space of the common factors and n unique factors. The vector 
representation of any variable in this space is given by 


zi: (Qin Gin + - -a Qim O, < a =s 0 25 O, ...,0), 


* This theorem is given by L. L. Thurstone, Vectors of Mind, 1935, p. 72. 


Lu 
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where the prime is employed to indicate the linear description of the ob- 
served variable 2;. The first m coordinates are with respect to the common- 
factor axes, and the last n coordinates, consisting of only one value different 
from zero, are with respect to the unique-factor axes. For simplicity let it 
be assumed that the common factors are mutually orthogonal, and, as 
usual, the unique factors are orthogonal to all factors. Then the norm, or 
length, of such a vector, according to (3.7), is 


(3.45) N(@;) = Vd +... + dis +a? = 1. 


In other words, each of the vectors representing the variables in the total 
factor space is of unit length. The direction cosines of such a vector in this 
space are simply the coordinates of the end point. The cosine of the angle 
of inclination (¢;,) of two such vectors, zj and zj, then becomes 


m+n m 
(3.46) cos $f, = >» MOMS = D anan = rik 
a=1 s=1 


where X, and dj, denote the sets of direction cosines of zj and zf, respec- 
tively. Equation (3.46) shows that the reproduced correlation for any two 
variables is the cosine of the angle between their vectors in the total-factor 
space. The reproduced correlation r/, will approximate the observed corre- 
lation r;, to the extent that the linear representations of the variables are 
adequate. 

Now the final interpretation of the variables as vectors in the common- 


factor space can be made. The orthogonal projections of the n vectors from 
the total-factor space into the common-f 


defined to be the vectors representin 
vector may be denoted by 


actor space of m dimensions are 
g the variables in this subspace. Such a 


2j' : (aj, 0p... 03) . 


The coordinates of the end point of this vector are the same as the first m 
coordinates in the total-factor Space. This property holds even if the com- 
mon-factor axes are oblique, provided only 
orthogonal to the common-factor Space. It 
plicity that the common factors are uncorr 

A projected vector in the m-space is us 
the corresponding vector in the total-fact 
only if the variable has no unique varia 


will again be assumed for sim- 
elated. 

ually of smaller magnitude than 
or space, being of the same length 
nce. Likewise, the angles between 
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pairs of vectors in the common-factor space are smaller and, consequently, 
their cosines larger. The length of a vector zj’ in this space is given by 


(3.47) NG) = Vai, ah... d.h, 


that is, the square root of the communality A? of the variable. The direc- 
tion cosines of any two vectors z/' and z;’ in the common-factor space are 


given by 


(3.48) M TR Mia Gt G-L2.,m. 
Putting these values into (3.29), the cosine of the angle of inclination of 


these vectors becomes 


i x Qj ks 
(3.49) cos oj = MIME = TERT 


s=1 


It is obvious that the expression (3.49) is generally larger than that given 
by (3.46), being equal to it only when hjhx = 1. Hence the angles between 
vectors in the common-factor space are smaller than the corresponding 
angles in the total-factor space. . 

The problem of interpreting a reproduced correlation 7j; geometrically 
can be treated in the common-factor space. It is evident from (3.46) and 
(3.49) that 


(3.50) cos $f, = ie ó 
j 

M H . 
The cosine of the angle of separation of two vectors representing variables 
in the common-factor space may be referred to as the correlation corrected 
for uniqueness. In other words, the expression (3.50) would be the value 
of the reproduced correlation between j and k if these variables were free 
from unique variance. Solving (3.50) explicitly for the reproduced correla- 


tion, there results 
(3.51) rh, = hjh, cos bjk- 
Thus, the reproduced correlation between two variables is given by the 


scalar product of their vectors in the common-factor space. Of course, the 
observed correlation r; differs slightly from the value given in (3.51), unless 


the residual is exactly zero. 
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A simple illustration of the foregoing ideas may be given for the case E 
only two factors. The common-factor space is of two dimensions, and t le 
two (uncorrelated) factors F, and F; are represented in Figure 3.3 by unit 
vectors separated by a right angle. Each variable z; of a set can be described 


mul 


Z2:(02,, 022] 


!Zi:(a,, Qia) 


Fia. 3.3 


in terms of the two common factors and a unique factor. The linear ex- 
pressions for two such variables may be written as follows: 


(8.52) r 1d, mit testi’ wht th d, 
i % = anFy + ax Fs +0 - Ui + asU;. 


* 
The geometric representation of 


these linear expressions for the original 
variables can be made in the tota 


-factor space of four dimensions, defined 
by the two common factors and t 


he two unique factors. In this space the 
vectors representing z/ and 2; are of unit length, and their correlation is 
given by 


(3.53) Tie = anan *l- Anay . 


All the essential information 
tions, can be obtained fro 
The projections of the tw 


about the variables, 
m the consideration of the 
© vectors z; and z/ 


i.e., the intercorrela- 
common-factor space. 
into this space are indicated in 
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Figure 3.3 by zj' and 23’, respectively, and may be written analytically in 
the form 


= zi = an, + avks, 
3.54 
men M ms 


nN 
3 
| 


The lengths of these vectors are given by the square roots of their commu- 
nalities, i.e., 
| D(Oz{') = Va, + aj; = Mh, 


3.55 eo E E. 
E" (D(Oz/) = Vai + ai, = Via. 


The cosine of the angle (6/2) separating these vectors is given by formula 
(3.49), as follows: 


n_a aa 0s de 1. 
(3.56) cos $12 = ur, Tr T (anan + ana») , 
3 
or, employing (3.53), y 
(3.57) Tia = hjh, cos $43 « 


This formula shows that the reproduced correlation of two variables is 
given by the product of the lengths of the two vectors by the cosine of their 
angle of separation in the common-factor space. . 

In the foregoing discussion it was necessary to employ distinct notation 
to clearly represent elements in the different spaces. Since it would be rather 
clumsy to retain primes and double primes in the remainder of this volume, 
they will be dropped when no confusion can arise as to the particular space 
involved. 


CHAPTER IV 


DETERMINATION OF THE COMMON-FACTOR SPACE 
4.1. Introduction 


In the preceding chapter it has been shown that the number of common 
factors necessary to describe a set of variables is equal to the rank of the 
correlation matrix. For a given set the observed correlations are known, 
but the communalities are unknown. The rank of the correlation matrix 
may then vary with the values put in the principal diagonal. It is thus pos- 
sible to determine values for the communalities which make the rank of the 
correlation matrix as low as possible. This produces a minimum number of 
common factors. 

The minimum number of factors for a given matrix of linearly independ- 
ent correlations is obtained theoretically in 4.2. It is then shown that, when 
certain relations exist among the correlations,* the set of variables can be 
described in terms of a smaller number of factors. These relations are de- 
veloped explicitly for one factor and any number of variables in 4.3, and 
for two factors in 4.4. The general procedure for obtaining such conditions 
among the correlations for any number of factors is indicated in 4.5. 

When the necessary conditions among the correlations of a se 
ables are satisfied for a certain rank of the matrix, 
be evaluated consistently with this rank. This pr 
4.6 and illustrated with a set of eight physical 
determining the communalities forms the theor 
tion of communalities discussed in 7.2. Such es 
principal-factor and centroid solutions which ar 
and VIII. Finally, to illustrate some of the me 
trix of correlations of six hypothetical variabl 
important uses of this cha 
sis in Chapter VI. 


t of vari- 
the communalities can 
oblem is considered in 
variables. This method of 
etical basis for the estima- 
timates are required in the 
e presented in Chapters VII 
thods of this chapter, a ma- 
es is introduced. Even more 
pter, however, are made in the methods of analy- 


4.2. Number of Common Factors 


One of the major 


problems in factor analysis is to determine how much 
the rank of the corr 


elation matrix can be reduced by a suitable choice of 


* When these relations exist, the property of linear independence does not hold. 
64 
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the communalities when the intercorrelations are given. The correlation 
matrix may be written 


h$ Ti Tus o... Tin 
Ta AR Tas scs Ton 
A = || ra Ta hi e T3n lly 
2 
fui Rae URS ee d 


where the communalities, h?, are unknown, and the observed correlations 
satisfy the property, rj; = ri; (j, k = 1, 2, ... , n). The rank of the sym- 
metric matrix A is m if an m-rowed principal minor Amm is not zero and 
if zero is the value of every principal minor obtained by annexing to Amm one 
row and the same column of A, and also of every principal minor obtained by 
annexing two rows and the same two columns.* By means of this theorem the 
number of linearly independent conditions that the unknown communalities 
must satisfy in order that the matrix A shall be of rank m can be determined. 

There are n rows in A and m in the nonvanishing principal minor Amm- 
This leaves (n — m) rows which may be annexed, one at a time, to Amm, or 
(n — m) determinants which must vanish. The (n — m) rows may be added 


two at a time in (" 2 Fi ways to Amm, giving (n — m)(n — m — 1)/2 
additional determinants which must vanish. Hence the total number of 
independent conditions (i.e., the number of minors set equal to zero) that 


the communalities must satisfy in order that A be of rank m ist 


— — m -— — — 1 
(4.1) V—— T5 (n E m—1) (n um. m + 1) . 
In general, the vm equations have solutions hj, A2, . . . , h? only if the num- 


ber of unknowns is greater than or equal to this number of conditions. If 
the number of unknowns is less than the number of equations, then the 
coefficients in the equations (the correlations) must satisfy certain relations 
in order that the number of independent conditions for the unknowns may 
be reduced to the number of unknowns. 


* L. E. Dickson, Modern Algebraic Theories, Theorem 15, p. 79. 


1 The proof given here is incomplete in the sense that the v, equations have not been 
shown to be linearly independent, i.e., that none of them follows from the others. Walter 
Ledermann in his article, “On the Rank of the Reduced Correlational Matrix in Multiple- 
Factor Analysis" (Psychometrika, 1937, pp. 85-93), arrives at the same number of con- 
ditions for the n unknown communalities, although by a different argument, and offers 
& proof of the linear independence of these equations. 


^ 
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First, however, let it be assumed that the correlations are arbitrary 
? H H 
values, then the set of conditions can be satisfied only if 


(4.2) n EY. 


The last inequality may be written in the following equivalent form: 


(4.3) Co m? — (2n + 2a + n(n — 1) <0. 


Setting the quadratic (m) equal to zero and solving for m, the two roots 
are given by 


(4.4) p= ED VT 


Now, it can readily be shown that the plot of the quadratic function $(m) 
is à parabola which opens up vertically. This curve crosses the m-axis at 
the two points whose abscissas are given in (4.4), and hence (m) < 0 for 
values of m between these extremes. The rank of the correlation matrix, 
with unknown communalities and arbitrary correlations, may thus be re- 
duced to the value m, which is given by 


(2n +1) + V8n +1 (2n + 1) — V8n +1 
(4.5) = — Lr mE 


" The smallest possible value for m is th 
or equal to the value in the right: 
is listed the smallest rank that c 
up to n = 15, when the correla 

E] 


en the smallest integer greater than 
-hand member of (4.5). In Table 4.1 there 
an be attained for a matrix of a given order, 
lions are assumed to be quite arbitrary. 


TABLE 4.1 
MINIMUM RANK UNDER ASSUMPTION Or " 
INDEPENDENT CORRELATIONS " 
Heese à Er? E EEY 0E. | 
[MN [*159946 à FEE Su a 


1 


Generally, the observed correlations from stati 
considered as arbitrary or independent. Th 
be reversed, that is, the number of unknow. 
of conditions which th 


stical variables cannot be 
e inequality in (4.2) may then 
ns may be less than the number 
ey must satisfy. The unknown communalities are 
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then 'overdetermined" in the sense that the larger number of equations 
may not be consistent. In order for a solution to exist, the coefficients in the 
equations must satisfy at least ¢(m) relations. The differences (v, — n), 
that is, the number of conditions that the correlations must satisfy so that 
a matrix of order n can be reduced to rank m, are given in Table 4.2. 


TABLE 42 
NUMBER OF INDEPENDENT CONDITIONS ON THE CORRELATIONS: é(m) 
a 
2 3 4 5 6 7 8 9 10| 11) 12] n 
= 
1....J-1} 0| 2| 5| 9| 14| 20| 27,35 44/54 (3?) -1-:6:3 
2....-2|-2 —1| 1| 4 s| 13| 19/26/3443 (3-2 n(1—9 1 
3 -a-3-2| ol 3| 7 1218/25/33 ae 3) - 3220P +8 
= -9 
Beclean aa 2| 90111724 (55) - 4-672 ts 
= =] 
T E. E E E EE (35) - 5-55 ) +10 
n— 6 |. n(n—18). 
6.. 4) 9 ( ;)-:*-4 15 
-— Ninus 
m. " pm -m= D + (7) 


The lower left-hand corner of the table has no entries because the rank 


Cannot exceed the order of a matrix. A negative value represents a larger 
number knowns than conditions, so that there is an infinite number of 
solution: uch a case, the general solution involving (n — vm) arbitrary 


paramieters. A zero value represents the case of as many unknowns as equa- 
tions. For a given number of variables, n, a negative or zero entry indicates 
the rank m which the correlation matrix can attain without any restrictions 
on the correlations. In these cases the inequality (4.2) is satisfied, and the 
conditions on the communalities are met under the assumption that the 
correlations are independent variables. The value of m for the first nega- 
tive or zero entry, reading down a column of Table 4.2, corresponds to the 


value of m, for the same n, in Table 4.1. 
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It is important for factor analysis, however, to attain a lower rank, for 
any value of n, than that given in Table 4.1. In an actual factor study the 
investigator should select the variables on some hypothesis of underlying 
factors.* Then the correlation coefficients will not be independent, and, 
owing to the consequent relationships among them, the rank of the corre- 
lation matrix will be lower than otherwise. In other words, the number of 
factors, m, necessary to describe a set of n variables will be lower than the 
minimum value given in (4.5) if the variables are selected to fall into sub- 
groups. This, of course, is desirable in factor analysis. If ten factors were 
necessary to describe fifteen variables, there would be very little gain in 
parsimony of thought. 


4.3. Conditions for One Common Factor 


The number of independent relationships that must exist among the cor- 
relations in order that the rank shall be lower than the minimum in (4.5) is 
given by the positive values in Table 4.2. Thus for n — 3, no relationships 
are necessary to attain rank one; that is, three variables can always be de- 
scribed in terms of one common factor. Four variables, however, cannot be 
described by just one factor unless their intercorrelations satisfy two (inde- 
pendent) conditions. These well-known conditions} are the vanishing of 
the tetrads, namely, 


(4.6) = = Tur = 0, 


Tiaoa — Tigres = 0. 


It may be well to indicate how the c 
n = 4 and m = 1, so that the m 
dent. When there are just four 
may be written 


onditions (4.6) are arrived at when 
ethod of generalization will be more evi- 
variables, the matrix of intercorrelations 


Tis. Ti 
Tn h mw r 

Ape lj Ta 24 
Ta Ta hj ra 


Ta Ta Ta bd 


This matrix will be of rank one if all second- 
selection of appropriate minors, several linear 
for the solution of each of the communalities, 
of the following three equations: 


order minors vanish. By the 
equations can be determined 
"Thus, h? is given by any one 


hor 2 
t Ti ao : hona si T3 Tij 0 
T US "e 
?1 T23 To Ta fa Tai d 
* See 2.7. 


T Charles Spearman, The Abilities of Man, 1927. 
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or 


(4.7) h = 


Trelis — Tilia — Tiaris 


T23 Tos T34 


On eliminating Aj, the two equations (4.6) arise. The solutions (4.7) for h? 
are consistent if the conditions (4.6) are satisfied. 

The question arises whether new conditions are introduced when the 
other communalities are solved for and eliminated. According to Table 4.2, 
only two independent conditions on the correlations are necessary to assure 
rank one for a fourth-order correlation matrix. To illustrate the dependence 
of any other conditions on the two already obtained, the solutions for the 
second communality are given: 

(4.8) h = 


TewWo3 — Teves _ Teales 


ns Tis T34 


These equations may be obtained by setting appropriate second-order 
minors of A, equal to zero or, more simply, by interchanging the indices 1 
and 2 in (4.7). On eliminating A3 from (4.8), the following conditions arise: 


(4.9) 


Tis 24 — Tiros = 0, 
rm — rra = 0. 
The first of these equations is identical with the second of equations (4.6), 
and the second is the difference of equations (4.6). Hence no new (independ- 


ent) conditions are introduced. . . . 
Referring to Table 4.2 again, it is seen that five relationships must exist 


among the intercorrelations of five variables if they are to be described in 
terms of only one common factor. The correlation matrix, 

hi Ti Tig Tia Tis 

T21 hi Toa T24 T25 
As = || r3 T» hi T3435 || > 

Ta Tay Tas hi Tas 

Ts Ts2 Toa T54 hi 
Will be of rank one if all second-order minors vanish. The five independent 
conditions that the correlations must satisfy may be obtained by the pre- 


ceding process. " 

Several linear equations for the solution of each of the communalities can 
be obtained by setting appropriate second-order minors of As equal to zero. 
The first communality, for example. is given by any one of the six equations: 


2 2 2 
hon] 0 hon. 0, h ns) 0, 
Tar T23 $ Ta Ta To T25 
2 
2 E r h rs 
hona aj. 1 52, i : =Ü 
Tai T34 Ta T35 41 T45 


70 FACTOR ANALYSIS 


or 


T; T1317) Tir; 
(4.10) a Tuis _ Tiu. — Tiris = 73714 0 Talis Mr 
s 1 T23 To4 Tos T34 Tas Tas 


In order that the solution for A? shall be unique, the following five condi- 
tions must be satisfied: 


(4 11) Tifis _ fiia _ Tiris = T3 Tuus — Tiaris 
i T23 T24 T25 T34 T35 


T45 


These conditions may be put in the equivalent form: 


T1324 — Tiaro = 0, 
T13725 — Tisfo3 = 0, 
(4.12) TiTa4 — Tir = 0, 
Tress — Tistos = 0, 


Tiaras — Turss = 0. 


Any other conditions must be linearly dependent on the above equations. 
Thus if instead of obtaining the solutions for hi, those for A2 were obtained, 
the resulting five conditions could then be Shown to be dependent on the 


foregoing relations.* A similar argument holds for the conditions obtained 
by solving and eliminating any other communality. 


* As in the case of h?, there are Six linear equations for the solution of hj, namely, 


= Tafa _ TaT = 72725 _ Tau _ Taas Tuna 
This Tis Tis Tas Tss Ta ` 


On eliminating A2, 


© 


the consistency conditions may be put in the form: 


Tais — Tuna = P 
(ii) Tali — Tuna = 
1) wis = 0, 
(iii) To — rana = 0, 
(iv) Tus — Tur = 0, 
(v) 


Taas — Tears = 


Equations (i) and (ii) are equivalent to i ion (iii 
respectively. t iii) 
is the difference of (4.12,) and (4.12,), Pe da a n e 
(4.124). To show that (v) is linearly d n (4.12), Substitute ri, = T1aTa3/T24, 0b- 
tained from (4.12;), into (4.12;). The result is 


TiaToTas — Tiu = 0 " 


which reduces to (v) by factoring out Tie 
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According to Table 4.2, a set of n variables can be described in terms of 
just one common factor if n(n — 3)/2 linearly independent relationships 
exist among the correlations. These conditions, whatever form they take, 
are equivalent to the following set: 


Tiris — Mii Lo Tin _ DU 
T23 Toà d Ton T34 


(4.13) 


For any variable ze a term of the form 
jy fett aedes Nie 
E exjx#k 


n—1 " 1 
will be called a ériad. It is readily seen that there are ( 2 ) triads in 


(4.13), or 


(4.14) i " j -1- sho 


equations of condition for one general factor among n variables. 

The number of conditions of the form (4.13) to determine whether a ma- 
trix is of rank one is considerably less than the number of tetrads. Every 
four variables give rise to three tetrads so that the total number of different 


tetrads for n vagipbles is 


(4.15) 


n n(n — 1)(n — 2)(n — 3) 
3 ( A à 


The difference between (4.15) and (4.14) is 


(4.16) pez 3| > [e a -— J , 


To indicate the magnitude of this number, suppose n = 15. The total 
number of tetrads is 4,095, while the number of triad conditions (4.13) 
is only 90. The difference (4.16) is 4,005. In other words, the labor of com- 
puting the conditions (4.13) is only about one-fortieth of that of computing 
the tetrads, for fifteen variables. For a larger number of variables the rela- 
tive economy of labor becomes more pronounced. 
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4.4. Conditions for Two Common Factors 


The necessary conditions for a matrix of correlations to attain rank two 
will next be considered. For five variables it is a well-known fact that one 
relationship must exist, namely, the following pentad criterion :* 


Tizl ea? 347 457 51 
+ rarest 340 aT 53 
(4.17) 


+ rarest air as so 


H+ risroar aor ass 


T123 asl 47 54 — Meal as 43% ii + iol oa ai asl 83 
Visas! 317 437 54 — T1324 gal 47 52 + TisTosTa4T aa 51 


ial esl 327 437 51 — 151237 aT aol 54 


0. 


This condition may be obtained by the method employed before. The cor- 
relation matrix, As, must have every third-order minor equal to zero if 
itis to be of rank two. By selecting appropriate minors, several linear 
equations for the solution of each of the communalities and hence the con- 


ditions for consistency can be obtained. Accordingly, for h? the following 
two determinants are employed: 


hi Ti3 Tis hi Tis Tis 
Ta Tos Tes) — O, Ta Tes Tes} = 0. 
Ta Tas Tas T31 T34 T35 


Equating the two solutions for hi, there results 


Ta (Tatas — T1843) — ra(risres — Tisl'as) 
(raras — Tost 43) 


= Ta(rurss — rura) — ra(riares — rigrod 
’ 
(raras — T2534) 


or 


[rers — Tessa] [rm (Taras = Tista) — Ta(risrog — Tis723)] 


= [rearas = Test aallraa(rargs = Tis?34) 


(4.18) ( 
= ra(rura — Twrai)] = 0, 


as the single consistency condition. If the intere: 
satisfy (4.18), or the equivalent condition 
can be determined to make the rank of A; equal to two, i.e., the five vari- 
ables can be described in term: 


According to Table 4.2, the 


orrelations of five variables 


* It is believed that this condi 


t tion was first obta 
the Mind of Man, p. 58; see also 


ined by T. L. Kelley, Crossroads in 
Ledermann, op. cit 
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four independent conditions in order for the matrix Ag to attain rank two. 
It will be convenient to make the following definition: 

hi Te Tu 
(4.19) |hi Ta ral = |T Te Teil x 


Tar Te Ted 


that is, a determinant is represented by the elements of the principal di- 
agonal, 

In order that Ag be of rank two, every third-order minor must vanish; 
five such determinants are 


|^i res ra| = |h} ces rae] = |h} ces css] = |h} ce rsel 


= | he T34 Tsel =0. 


The solutions for h? from these five equations are as follows: 


AP Taffia Ta4s| — TulTis T25 
A [r23 Tas| P 
pea Ta [T13 Tacl — Tafris T26 
: [res Tael ' 
Talis Ts6| — Tar|713 126 
(4.20) k = : 
[r23 "sc 
= Talia Tse| — TsiiTi4 126 
i= E 
| 724 Tsel 
hea Ta|Ti4  756| — TwulTi 736 
: fras Tse| g 
where 
Tab Tad 
(4.21) [rab Tea | = = Tabled — Tadlcd - 
Teb Ted 


Eliminating h? from the five equations (4.20), the four conditions which 
the correlations must satisfy are obtained. The equality of the right-hand 
members of equations (4.20) are then the necessary conditions for six vari- 
ables to be describable in terms of two common factors. — A 

This process can be continued for any number of variables. The posi- 
tive entries for m = 2 in Table 4.2 give the number of conditions that must 
exist among the correlations for a set of n variables to be describable in 
terms of only two common factors. In general, the correlations among n 
variables must satisfy (n? — 5n + 2)/2 conditions in order that their ma- 
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trix shall be of rank two. These conditions may be obtained by eliminating 
any communality A? from (n? — 5n + 4)/2 equations of the form 


7, |r Tea | = Yee] tros Tad | 
pe = To e 
Cpa : [Fas al 


idee deis ML 
esasbsecsd 


It will be noted that many more conditions than the number indi- 
cated in Table 4.2 can be written for the intercorrelations of » variables 
by means of the foregoing procedure. Corresponding to any four indices 
in the denominator of (4.22) there is a third-order determinant of the 
form (4.19) which is to be set equal to zero for the calculation of a particu- 


à ; —] 
lar communality A2. Out of a given set of n variables there are ( ” 
(n — 1)/(n — 5)!4! 


choices for the denominator, and each of these may 
be permuted in 4! ways. The total number of possible denominators, and 
hence, third-order determinants, for the calculation of the communalities 
would seem to be enormous. Fortunately, however, this number is con- 
siderably reduced owing to the Symmetry of the correlation matrix and 
certain properties of second-order determinants, 

Although there are 24 possible 
8s many determinants (4.21), 
nants have only two different 


(4.23) lab cd| = 
(4.24) 


ry 
e 
e 

ie 
D 


—|cb ad], 


where, for simplicity, only the indices of the correlation coefficients are 
written. Then the two values of (4.21) for any four indices a, b, c, d arise 
from the successive applications of (4.23) and (4.24), as follows: 


(4.25) |ab cd| = |ba de| = — |da be| = —lad cb| = jed ab] 
=|de bal = — lbe da| = = |cb ad|, 

(4.26) |ab de| = |ba BH = = fee Bd] me — [ao db| = |dce ab| 
= |ed baj = — |bd ca] = — |db ac| 


i resulting f. er- 
mutation of the four indi i / "AB a another p 


expressions, one 
from (4.25) and the other from (4.26). For example, [ac bdl = [ab cd] 
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— |ab dc|, which may be checked by writing these determinants in full 
and making use of the symmetric property of correlations. 

The evaluations of à communality (4.22) which are based upon the de- 
nominators indicated in (4.25) are all equal.* These equalities are quite in- 
dependent of the assumed rank of the correlation matrix. The investigator 
should employ thus only one of the equivalent denominators in (4.25). If 
he should employ all eight, and obtain eight identical evaluations of the 
communality, he would still have no assurance that the assumed rank of 
two for the correlation matrix is correct. The values of the communality 
employing the denominators in (4.26) are, again, all equal, but usually dif- 
ferent from the value involving a denominator of (4.25). If one of the re- 
maining eight permutations of four indices is employed, it can be shown that 
the resulting evaluation of the communality is a direct result of the pre- 
ceding two evaluations. 

To establish this property, consider the two evaluations of h? dependent 
upon the denominators of (4.25) and (4.26), respectively: 


EN. leb cd| — celeb ad| 
D. jab cd| 1 

p _ N’ _ aeleb dc| — de|eb ac| 

ado Ey Jab de] , 


where E and E' are used to represent the particular evaluations, and only 
the indices of the correlation coefficients are written for simplicity. Consider 
also the evaluation employing |ac bd| in the denominator, namely, 


RB" = N” _ aelec bd| — belec ad| 
D” {ac bd| 


* The equality of the eight evaluations of a particular communality employing the 
denominators of (4.25) is indicated in the following manner. Let E be the evaluation of 
a communality A? based upon the first determinant of (4.25), namely, 

a= Tae | ted Ted| — Tee |Teb Toal 
E [rao Tea | 
The evaluation based upon the second determinant of (4.25) is 
go [Tea Tac |. — rael Tea Tre] 
25 [ros rac] 
Tee(Teated — Teo) — Tae (Teac — TecTba) 
= TbaTde — TbcTda 


Tea(TbeTed — TdeTbe) — Tec(TbeTda — TeeTbo) ; 
Tabled — Tead 


which is evidently equal to E. As one further illustration consider the evaluation based 
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It was indicated before that |ac bd| — |ab cd| — |ab dc|, that is, D" — 
D — D'. By expanding the determinants in the numerators, it follows that 


N- N 


Tae(TebTed — Tara) — Te(raraa — Tabled) 

— Tae(TesPac — rare) + Ta (rarae — rare) 
= Tae(TecTod — Tere) — Toe(TecTad — rara) 
ae|ec bd| —belec ad| = N”. 


Hence the evaluation E" can be expressed in terms of the other two evalu- 
ations as follows: 
EU N" .N-N' ED cC- E'D' 
D" I»—p' ^ D- p 


If the evaluations E and E' are reasonably close to the true value A2, then 
E" is also equal to this value, for, setting E = E' = k, it follows that 
h?(D — D) 
ne fhe iam 
E" = pop =. 


The preceding discussion indicates that, of the twenty-four evalua- 
tions of a particular communality from the permutations of four vari- 
ables selected for the denominator of (4.22), only two determinations 
need be considered. For a set of n variables there are then 2(” — 4 
(n — 1)!/12(n — 5) 
that the rank of the 


upon the last arrangement of the indices a, b, c, din (4.25), that is 
, ] 


E” = Tel res Taal — Tae |Te& Tea | 
Lee leb Tod] — Tae|Tes rea | 


Ires raa] P 
which, on applying (4.24) to the denominator, becomes 
E" = = iela tal = Tee |Teb Taal] 
— [ras Tea = 


Similarly, any other evaluation loyi P 
be equal to E. employing the denominators of (4.25) can be shown to 
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4.b. Conditions for m Common Factors 


In the general case, the correlation matrix A will be of rank m if every 
(m + 1)-order minor vanishes. By the selection of appropriate minors, a 
number of linear equations can be obtained for the solution of each of the 
communalities. Then, by eliminating the communality, there results the 
conditions that the correlations must satisfy in order for A to be of rank m. 
A typical equation involving Aj linearly is 


hi 13 ns ssw: se) s — Thimgd 
T21 T23 T25 e. Te2k—l es To 2m41 
Tu T43 Tas eee Task. -ee TA ml 
D= =0 
Taja Tea Tops +++ T2j2k-1 +++ T2j,2m+ 
Temi T2m,3 Tam,5 = ++ T2m,2k-1 +++ T2m,2m+1 


It may be noted that the (j + 1)st row is given by the first index equal to 
2j and the kth column by the second index equal to 2k — 1. The determi- 
nant D is thus seen to contain (m + 1) rows and (m + 1) columns. 

The determinant D may be expanded according to the elements of the 


first column, as follows: 
Du =a Da Hra Dat cs: (— 1), Do1 T... (— 1)?ren, 1Dam =0, 


where 
Dy, = minor of the element Aj , 
Dj, = minor of the element rj; . 


The communality of the first variable is then given by 
1 3 —1)-Ip i 
(4.27) =p 2 1)*7tro, Doi . 


This solution involves the correlations among the first (2m + 1) variables, 
and, except for the first one, the correlations of an even-numbered variable 
are only with odd-numbered ones, and odd only with even. Other determi- 
nants than D, still employing only the first (2m 4- 1) variables, may be 
equated to zero to yield linear equations in hi only. Then different variables 
may be introduced from the original set of n to obtain still other pm 
equations in 2. Thus the requisite number of equations of the form (4. 7) 
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can be obtained, and from them, in turn, necessary restrictions on the corre- 
lations. The number of such independent conditions is given in Table 4.2. 


In the general case of n variables and m factors, the number of conditions 
is given by 


o USE NGEB UMEN In 


so that there should be one more than this number of equations of the form 
(4.27). Actually many more equations of this form can be written, just as 
in the case of rank two. Again, however, it may be remarked that a good 
approximation can be obtained by employing the indicated number of ex- 
pressions (4.27). 


In the preceding analysis it has been tacitly assumed that 
nz2m--1, 


so that a determinant of the t 


ype D, involving the correlations among 
(2m + 1) variables, 


would be possible. If there were not this number of 
variables, then some other procedure would be necessary, for it would not 
be possible to obtain linear equations in only one communality. Perhaps 
the best procedure then would be to select (m + 1)-order determinants in- 
volving the smallest number of com: i 


4.6. Determination of Communality 
i Up to this point various conditions for a correlation matrix to have a 
given rank were set down. These conditions were obtained by equating & 


ular diagonal entry or communality. It 
e could be employed in determining the 


ere known. Although the rank of a correlation 
directly, it may be a 
rmined b 


is then evident that this procedur 
communalites if the rank w 
matrix cannot be obtained 
of groups of variables, dete 
The number of such groups is appro 
independent factors, and these in 
common-factor space. Then the 
rank of the correlation matrix is equal to the number of dimensions of the 
Space containing the vectors rep 
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In an actual analysis the number of common factors (or the rank of the 
correlation matrix) may be assumed to be equal to the number of groups G.. 
Then the methods of the preceding sections may be applied to check 
whether the correlations satisfy the necessary conditions for their matrix 
to be of the assumed rank. If the correlations satisfy these conditions, then 
the communalities are given by equations of the form (4.22) for rank two, 
or, in general, by (4.27) for rank m. As will be indicated, the order of pro- 
cedure is to set down a number of evaluations of each communality for the 
assumed rank. The consistency of these values will serve as a check on the 
assumed rank, and their average may then be taken as the appropriate 
communality. Similar determinations would be required for each variable 
in the set. The assumed rank must check for all such determinations of 
the communalities. 

Of course, the correlations need not satisfy the conditions for a given 
rank exactly with actual data because allowance must be made for chance 
errors. A sampling error formula for the general expression in the right- 
hand member of (4.27) is not known at the present time. The standard error 
of the simplest instance of this expression (the triad), however, is developed 
in 6.5. When the rank of a matrix of correlations is assumed to be one, the 
true values of the various triads for the communality of any variable are 
equal. Thus by means of the standard error formula for triads, it is possible 
to determine whether the variations of the obtained values are attributable 
entirely to fluctuations in sampling. It is suggested that for rank one all 
possible triads be written in the calculation of a particular communality. If 
the variation among these values can reasonably be assigned to chance fluc- 
tuations, the mean value may be taken as the communality. . 

In the case of rank two, all possible expressions (4.22) for the determina- 
tion of each communality could be considered. Before being averaged, how- 
ever, those based upon insignificant denominators would need to be rejected. 
The variables yielding insignificant tetrads for the denominator of (4.22) 
can be identified when the design of the variables is known, and = are 
but two groups. In such a case each group of variables will MN 
rank one, and the tetrads involving three variables of such a pue will be 
insignificant. Knowing the combinations of variables which produce in- 
significant denominators, it is not necessary to consider the i MU 
(4.22) which involve them. The denominators should Wee two lies uw 
of each group, considerably reducing the total number of expressio 


each communality. . = 

When the wank of a correlation matrix is assumed, and the d 
of the communalities is attempted, it may sometimes happen t ae 
the values exceed unity. Of course, such values of the communalhti 
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not permitted, and they indicate that the assumption of the rank is in- 
exact. Before the hypothesis of the specified rank is discarded, however, a 
number of evaluations of communality should be attempted. If, in general, 
several consistent values for each communality can be obtained, they should 
be averaged for the best determination of the communality, ignoring those 
values which exceed unity. The justification for this procedure lies in the 
fact that the observed correlations are themselves subject to error, and the 
values to be supplied in the diagonal of the correlation matrix to produce a 
specified rank can only be expected to satisfy this hypothesis approximately. 
The final check lies in the agreement of the reproduced correlations, from 
the solution employing these communalities, with the observed correlations. 


TABLE 4.3 


INTERCORRELATIONS OF EIGHT PHYSICAL VARIABLES 
FOR 305 FIFTEEN-YEAR-OLD GIRLS 


Variable 1 2 3 4 5 6 7 8 
T; GRBs in ss wae ip nl ao eee [veo c ddr e coe 
2. Arm span... .......... eee 
3. Length of forearm... ... 05 881 ee 
4. Length of lower leg..... 859 | .826 : . 
b; Weight. eias a aen -473 | .376 | . ABD Tos zs s [acci ers Psion ac rece oe 
6. Bitrochanteric diameter.| .398 -826 | .319 | .329 | .762 |... osse 
7. Chest girth............ -301 | .277 | .237 | .327 xU. || OBR: levee axes octo ien 
8. Chest width 


PPE -382 | .415 | .345 | .365 .629 | .577 | .539 


If the final residuals are insi 


gnificant, then the choice of the communalities 
is statistically sound. 


of “lankiness” and the latter 


' This design is substantiated by the meth- 
ound that 


four as measures of “stockiness,’ 
od of B-coefficients, It is thus f 


B, 2, 3, 4) = 235 " 
B(5, 6, 7,8) = 179. 
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Assuming rank two, the communality of any variable z, can be obtained 
by averaging a number of evaluations of (4.22). The calculation of such 
expressions can be facilitated by organizing the work as in Table 4.4, where 
the communality for the first variable in the given set is determined. 


TABLE 4.4 
CALCULATION OF h? 
ab cd ra|ris re] — rer|rib roal N D Ar 
.846(.4622) — .473(— .0882) .4327 .5474 .7905 
.846(.4733) — .473(— .0422) .4204 .5379 .7815 
.846(.3612) — .473( — .0025) .3068 .3964 -7739 
.846(.3733) — .398( — . 0422) .3326 .4253 .7821 
.846(.3426) — .398( — .0025) .2908 .3760 -7735 
.846(.3434) — .301( — . 0025) .2913 .3765 . 7736 
.846(. 4810) — . 473( — .0487) . 4300 .4873 .8823 
.846(. 4958) — .473( — .0107) .4245 .4822 .8804 
.846(.3738) —.473(  .0410) .2968 .3386 .8767 
.846(.4018) — .398( — .0107) .3442 .3904 .8816 
.846(.3700) —.398( .0410) .2967 .3401 .8724 
.846(.3381) —.301( .0410) .2737 .3095 .8843 
.805(.4810) — .473(— .0448) .4084 .4713 .8665 
.805(.4958) — .473(— .0375) .4169 .4814 .8659 
.805(.3738) — .473( — .0096) .3054 .3534 .8643 
.805(.4018) — .398( — .0375) .9384 . 3890 .8699 
.805(.3700) — .398(— .0096) .3017 .9487 .8651 
.805(.3381) — .301(— .0096) .2751 .3189 .8625 
Loc dien ge: as cia oo n D AUN FU, Sae we Ae SINGERS .842 


In the stub of the table are recorded the four indices a, b, c, d which de- 
termine the denominator D of (4.22), namely, 


H= |re ral. 


These indices are selected so that two are included from each group, and 
duplications which arise by certain permutations, as indicated in (4.25) and 
(4.26), are omitted. Thus an orderly arrangement can be obtained by con- 
sidering all possible combinations of pairs of variables from the first group 
with every pair of variables from the second group. The variable for which 
the communality is being calculated must, of course, be omitted from such 
Pairings. In Table 4.4 each pair of variables 23, 24, and 34 is considered 
along with each pair 56, 57, 58, 67, 68, and 78. Eighteen distinct values of 
the denominator, and hence the expression (4.22), are thus considered in 
Table 4.4. It is true that eighteen additional determinations could be made 
by interchanging the variables in only one of the pairs ab or cd. Since the 
computed values in Table 4.4 check the rank closely, it is highly improbable 
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that the additional eighteen determinations would be inconsistent with the 
former, and hence they need not be calculated. 

When the indices in the stub have been set down, it is suggested that the 
denominators in column D be computed. Some unnecessary work might be 
avoided by doing this first. For, if the original design of the variables was 
not accurate, three of the four variables might actually belong to one group 
(or measure a single factor), and the resulting D would then be very small. 
Such a combination of four variables would be rejected from the calcula- 
tions, and no further work would be done in that row of the table. 

After the denominators have been computed, cross out any that are in- 


significant, and then obtain the numerators of (4.22). For the communality 
of the first variable these numerators are 


N=ralry teal —ralrs ral. 


In the tabular arrangement of the work it is well to record all the coefficients 
Ta and ra before calculating the second-order determinants. It is also de- 
sirable to calculate all the second-order determinants of the first column, 
and then those in the second column, rather than the two in each row. With 
this procedure the successive second-order determinants can be calculated 
from Table 4.3 in an orderly fashion, By performing the indicated multipli- 
cations and combination of terms in each row, the numerators are obtained. 
These are then divided by the respective denominators to get the eighteen 
values of h?. The consistency of these values, as shown below, indicates that 
the assumption of rank two was justified. The mean is then taken as the 


ases. The complete calculations for each 
be produced here, but the eighteen evalu- 


" No exact Standard for judging the consistency of these values is available, 
ut a ca for determining the maximum allowable variation from the 
mean value can be set down. The calculated communality, although a de- 
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rived statistic, may be regarded as a variance, and the usual formula for 
the standard error of a variance applied to it. This formula is given by 


2 
(4.28) Ge = NIAE 
As in the case of other formulas involving derived statistics, it is expected 
that the values obtained from (4.28) will be smaller than those which would 
arise from a true formula developed for the communality. Therefore, if the 
variations of a set of values for a communality from their mean can be 
shown to be insignificant by use of formula (4.28), they would also be in- 


TABLE 4.5 


DETERMINATION OF COMMUNALITIES FOR EIGHT 
PHYSICAL VARIABLES 


Al Aj DH h DH LH D LH 


.7905 | .9301 | .8347 | .8332 | .9503 | .6117 | .5599 | .4871 
“7315 | .9279 | .8354 | .8416 | .9518 | .6105 | .5656 | .4813 
“7739 | .9258 | .8357 | .8378 | .9567 | .6063 | .5595 | .4793 
“7821 | .9270 | .8369 | .8386 | .9538 | .6093 | .5613 | .488U 
"7735 | .9261 | .8373 | .8388 | .9547 | .6075 | .5589 | . 4842 
UTI36 | .9255 | .8416 | .8378 | .9567 | .6065 | .5593 .4808 
8823 | .8190 | .7534 | .8510 | .8297 | .7010 .6360 | .4889 
/8804 | .8147 | .7529 | .8561 | .8290 | .7023 | . 6476 | .4766 
“3767 | .8076 | .7513 | .8535 | .8268 | .7068 | . 6367 | .4722 
'8816 | .8133 | .7496 | .8541 | .8220 | .7075 | . 6544 | .4708 
"9724 | .8138 | .7504 | .8551 | .8079 | . 7224 | .6483 | . 

78843 | .8080 | .7526 | .8519 | .8209 | . 7116 | .6391 | .4735 
.8665 | .9090 | .8539 | .7552 | . 8482 | .6234 | .5484 | .5475 
“g659 | .9028 | .8590 | .7608 | . 8446 | .6224 | .5534 | .5401 
18643 | .8945 | .8663 | .7489 | .8481 | . 6247 | .5419 | .5408 
8699 | .9060 | .8565 | .7556 | .8242 | . 6127 | .5578 | .5431 
“g651 | .9036 | .8588 | .7500 | .8342 .6287 | .5397 | . 

“g625 | .8962 | .8715 | .7480 | .8439 | . 6262 | .5409 | .5435 


Communality PEN .842 .881 .817 .815 .872 .647 .584 .502 
Standard error....| .068 | .071 .066 | .066 | .071 .052 | .047 | .041 


est. The standard errors for the eight 


Significant by the more accurate t 
ormula, are presented in Table 4.5. 


communalities, as given by the above f 
It will be observed that the maximum variation from the mean in any col- 
umn does not exceed 1.5 times the standard error, clearly demonstrating 


the consistency of these values and justifying the assumed rank and the de- 


terminations of the communalities. 

The preceding analysis is based upon à knowledge of the design of the 
variables in a given set. In case the variables are selected without a particu- 
lar design, they might still be grouped from the nature of their correlations, 
and the same procedure applied in the calculation of the communalities. Tf 
such grouping of the variables is not feasible, then the suggested selection 


of variables (e.g., as in Table 4.4) may be modified. 
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Without knowledge of the grouping of variables, which guided fus sehen 
tion of the stub in Table 4.4, the indices might be selected on an arbitrary 
basis. One suggestion is to write a number of trials based upon variables 
selected with equal frequency from the whole set. An illustration of the 
stub of such a table for the calculation of h? from a total set of eight vari- 
ables is presented in Table 4.6. The total number of distinct trials for the 
determination of any communality from a set of eight variables is 

(n — 1)! 7! 
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12(n — 5)! ^ 12(3)) 


as indieated on page 76. The proposed Table 4.6 includes only special 
combinations of four variables, namely, four consecutive ones in the first 


seven trials and the corresponding four with the first pair interchanged in 
the last seven trials. 


TABLE 4.6 
STUB FOR VARIABLES WITHOUT DESIGN 


ab cd 
23 45 32 45 
45 67 54 67 
67 82 76 82 
82 34 28 34 
34 56 43 56 
56 78 65 78 
78 23 87 23 


If the assumed rank of two is verified by the evaluations from a set of 
indices such as those in Table 4.6, then th 


as the average of these evaluations. I 
the communality, the evaluations ba: 
must be rejected. Thus the actual nu 
ing the rank and computing the co 


e communality would be accepted 
n checking the rank, and computing 
sed upon insignificant denominators 
mber of evaluations retained for check- 
mmunality may be smaller than that 
indicated in such a table. A more refined check on the rank could be ob- 
tained by employing additional evaluations. 

The direct application of the foregoing method of computing communali- 
s is practical when m is equal to one or two. For a larger number of fac- 
tors the direct procedure becomes too cumbersome. The methods of this 


chapter, however, form the basis of analysis into any number of common 
factors (see Chap. VI). 


tie 


DETERMINATION OF THE COMMON-FACTOR SPACE 85 


4.7. Illustration of Direct Solution 
For u small set of variables, when the communalities have been computed 
by the method of the preceding section, it may be possible to obtain a direct 
solution for the factor pattern. Such a solution is possible if, in addition to 
the knowledge of the communalities, a pattern plan is also postulated. This 
type of analysis will be illustrated with a set of six hypothetical variables. 
Assume a matrix of intercorrelations of the six variables as follows: 


72.75  .49 .42 28 


72 78 42 36 .24 
35 78 35 .30 .20 
^-|49 42 35 42 28 
42 36 30 42 24 


|.28 .24 .20 .28 24 


The problem is to formulate the pattern plan, determine the six communali- 
ties, and caleulate the coefficients of the common factors. 

In order to compute the communalities, the approximate rank of A is 
required. This rank may be determined indirectly from the number of 
groups of variables. Applying the method of B-coefficients, it is found that 


B(1, 2,3) = 221, 
B(4, 5, 6) = 92. 


The variables 1, 2, 3 are thus seen to belong together while 4, 5, 6 do not 
belong together any more than to the preceding three variables. The fol- 
lowing groups are consistent with this analysis: 


Go = (1, 2, 3, 4, 5, 6), 
G, = (1, 2,3), 


and so the rank of A may be assumed to be two. A plausible factor pattern 
plan,* which takes these facts into account, Is glven In Table 4.7, where un- 
correlated factors are assumed for convenience. . . 
Now the four necessary conditions which the correlations must satisfy 
in order that the six variables shall be describable in terms of two ae 
factors are obtained by equating the right-hand members of equations (4.20). 


* The pattern plan is plausible because the correlat ions € d n: 
are higher than those among 4, 5, 6. Such a plan is consistent eq ga sapie on 
an extra factor should be postulated for a group of variables mee : high in ` 
This hypothesis is implicit in all current methods of factor analysis. 
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The value of each of the first three of these expressions is exactly .74, while 
the last two expressions are indeterminate.* The correlations satisfy the 
necessary conditions, and hence A is vf rank two. The communalities are 
then given by equation (4.22), for n -- 6, and the average of the various 
determinations of each communality may be taken. For the present hypo- 
thetical problem the different solutions for any communality yield the same 
value. The computed communalities are presented in Table 4.8. 


TABLE 4.7 
FACTOR PATTERN PLAN" 


Variable | Fo | Fi 


1 Qo an 
2.. 29 an 
3.. a30 an 
4.. ato 

5. a ik aso 
[ENS is 6 — [oe 


* A factor pattern will usually be presented 
in such a tabular form with the coefficients of 
the respective factors appearing in the columns 
headed by the factors. 


TABLE 4.8 
COMMUNALITIES OF SIX HYPOTHETICAL VARIABLES 


Variable 1 2 3 4 5 6 


HEC TS 74  .72  .89  .49 .36  .16 


From the intercorrelations and communalities, all the coefficients of the 


factor pattern can be obtained. Since the last three variables involve only 
one common factor, each of their commu 


1 nalities is merely the square of the 
coefficient of this factor. Hence 


hi = 49 = ah, 
or 


Qo = .7. 
Similarly, : 


dey = 6 and 


* This is due to the fact that the 
1, 2, or 3 vanish. As a consequence 


uch variables are describable in terms of only 
one common factor. 
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The coefficient ai; may be obtained by means of any one of the correla- 
tions ris rf, or rts; for, by (2.22), 


Tik = Upon (k = 4, 5, 6) . 


A similar argument holds for the determination of az and as. Then, em- 
ploying the mean of the three evaluations of any one of these coefficients, 


they are given by 
6 
1 Tjk " 
up ED. 6G =1, 2,3), 
k=4 


ro 


where the correlations from the factor pattern (the 7/,) are replaced by 
the observed correlations in A. The tacit assumption is that the residuals 
are zero. The coefficients of Fo for the first three variables have the follow- 
ing values: l 
Gig F's an = 6, ax = .5. 
Since the coefficient aio is known, the remaining coefficient an in the first 
equation of the pattern may be determined from hj. For 


hì = a + ah, 
or 


ay = Vh — at) = V.74 — 49 = 5. 


.8. The factor pattern thus computed from 


Similarly, a4 = .6 and aa = 
the correlations in A is given in Table 4.9. 


TABLE 4.9 
FACTOR PATTERN FOR SIX 
HYPOTHETICAL 
VARIABLES 
Variable | Fo Fi 
aT Jb 
.6 .6 
a) 8 
sT eee 
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It may readily be checked that the correlations reproduced from the 
pattern of Table 4.9 agree exactly with those given in A. Thus, 


Tj = (6) + .5(.6) = 72 = rg. 


In ordinary practice, of course, some discrepancies between the original and 
reproduced correlations would be expected. The approximate agreement be- 
tween these values is the essential requisite for the adoption of any factor 
solution, the form selected being somewhat arbitrary. This is illustrated in 
the present example by postulating the plan of solution of Table 4.7. A dis- 
cussion of various preferred pattern plans which may be employed for a 
given set of data will be taken up in the following chapter. 


CHAPTER V 
PREFERRED TYPES OF ORTHOGONAL SOLUTIONS 


5.1. Introduction 

In the choice of a scientific hypothesis two possibilities immediately arise 
from the fact that the factors may be taken as correlated or uncorrelated. 
This leads to two distinct developments of the theory. In this and the next 
five chapters the common factors will be assumed to be uncorrelated, or geo- 
metrically the axes representing them will be taken as orthogonal. Then in 
Chapter XI the case of correlated, or oblique, factors will be discussed. 

When it has been decided to describe a set of variables in terms of un- 
correlated factors, the problem is not yet unique. There are still an infinite 
number of orthogonal reference systems in terms of which the variables may 
be described. The next problem is to select one of these frames of refer- 
ence, on some rational basis, as a preferred system. Certain preferred types 
of orthogonal factor patterns will be discussed in this chapter, after a con- 
sideration of some basic standards for judging the satisfactoriness of form 
as a simple scientific description of the variables. 

The extent to which each of these types of solution conforms to the basic 
Statistical criteria will be discussed in detail. It is shown that by the intro- 
duction or elimination of one or more of these standards it is possible to pass 
from one preferred form to another. Thus it is hoped that some unification 


of factorial methods will be achieved. : 
In this chapter only schematic, or formal, solutions of the preferred types 


are presented. Although the algebraic equations of the various factor pat- 
terns are exhibited, the formulas for the computation of the coefficients are 
not given here. The special analytical methods employed in obtaining nu- 
merical solutions of the preferred types will be developed and illustrated in 


detail in later chapters. 


5.2. Some Standards for Judging Preferred Patterns 

In order to limit the infinitude of possible factor solutions that can be ob- 
tained in describing a given matrix of correlations, a set of restrictions must 
be imposed to obtain some preferred reference system. To this end, a list 
Of statistical standards is presented which is based partially upon those 
found useful in previous investigations. Some of these standards are analy- 
tical in character, while others are of a geometric nature. Inasmuch as both 

89 
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these types of standards are designed to produce simple forms of solution, 
certain of these standards are somewhat related. They are listed here sepa- 
rately, however, because they support one another and simplify the selec- 
tion of preferred solutions. By presenting such a broad list of criteria, the 


assumptions underlying each type of preferred solution can be exhibited 
explicitly. 


l. AGREEMENT WITH ASSUMED COMPOSITION OF VARIABLES 

The composition of variables postulated in Chapter II was based on 
three types of observation, namely, that correlations are found among vari- 
ables of a set, that potential linkages may also occur among variables in the 
set with others not included, and that all measurement is subject to error. 
These considerations led to the fundamental equations (2.8) involving com- 
mon, specific, and unreliable factors. All forms of solution should obviously 
conform to the factorial composition of the statistical variables postulated in 
such linear equations. 

2. PARSIMONY 

According to the principle of parsimony common to all branches of sci- 
ence, a law or description should be simpler than the data upon which it is 
based. This may be illustrated in the fitting of a theoretical curve to a series 
of observations. The number of constants in such a function should be much 
smaller than the number of observations in order to give a simple and useful 
interpretation of the latter. Similarly in a factor problem the functional de- 
scription of the variables should be much simpler than the original data. 


a) Number of common factors.—In agreement with this principle the total 
number of common factors should be considerab 
of descriptive variables com 


s principle of parsimony be 


; e linear description of each variable the 
complexity should be as small as possible. 


The advantages of a solution involving uncorrelated factors arise from 


PREFERRED TYPES OF ORTHOGONAL SOLUTIONS 91 


convenience of initial solution and subsequent interpretation of factors. As 
just indicated, an orthogonal solution is the fundamental form from which 
others may be derived. In some instances, however, such a solution may 
conform to a preferred type without further transformation. If uncorrelated 
factors are selected for the final solution, there is a decided clarity of in- 
terpretation, especially in the description of individuals in terms of factors. 
Such descriptions are clearer and more economical than if expressed in terms 
of interrelated factors. 
4. RELATIVE CONTRIBUTIONS OF FACTORS 

Another standard which may be useful in the selection of a particular 

type of solution is based upon the relative contributions of factors. As de- 


fined in 2.3, the contribution of a factor is given by the sum of the squares 
of its coefficients for all the variables. Three useful types of relationships 


between the contributions of a set of factors follow: 
a) Decreasing contributions.—In this case the various factors contribute 


successively smaller amounts to the total communality. 

b) Level contributions.—A second choice is one in which each factor con- 
tributes approximately an equal amount to the total communality. 

c) One large and remaining level contributions.—The third type of rela- 
tionship is that in which one factor contributes a large amount, while the 
remaining factors contribute a much less but fairly uniform amount to the 


total communality a 
: ion of a factor is related to its statisti- 


It can be shown that the contributi } 
cal significance. This has been demonstrated bot h by theoretical treatment 


and by empirical evidence. 


5. GEOMETRIC FIT: VECTOR REPRESENTATION 


As pointed out in Chapter III, the geometric interpretation of the factor 
Problem frequently adds clarity to the analytical method. This is especially 
g to several distinct types of 


true in formulating a set of standards leadin, i 
Dreferred patterns. When such distinctions have been made in geometric 
terms, the corresponding analytical properties of the factor patterns are 
explained, The following geometric criteria furnished the bases for several 


Scientifically desirable factor patterns. > 

a) Linear fit.—An obvious justification for postulating common factors 
1s that the variables in a particular investigation usually are correlated. 
Furthermore, certain subsets of variables may show generally higher inter- 


Correlations among themselves than with the remaining variables of the 
this type may then 


total set. Group factors corresponding to each cluster of 
€ assumed. In a sense, then, à group factor may be regarded as a sort of 
average or the common element of the variables of such a subset. 
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It has been shown in 3.7 and 3.8 that the variables may be regarded as 
vectors with a common origin and that the correlations between variables 
are given by the cosines of the angles between such vectors in the N-space, 
or by the scalar products of the projected vectors in the common-factor 
space. A group of variables yielding a cluster of high intercorrelations is 
thus encompassed by a “cone” with a relatively small generating angle. If 
a reference axis or vector of the common-factor space is chosen in the midst 
of this cone, all variables in the group will correlate high with it. To a fac- 
tor common to a group of variables, there thus corresponds a reference vec- 
tor. The degree of linear fit is measured by the compactness of the “cone” 
representing the particular group of variables, or the extent to which the 
vectors of these variables approach the axis of reference. By selecting a 
number of such reference axes, each one passing through a cone of vectors, 
the whole configuration may finally be well fitted. 

The standard of linear fit, together with that of uncorrelated factors, 
usually can be met only roughly in the case of positive correlations among 
the variables. It is evident that a closer linear fit can be obtained by per- 
mitting the factors to be correlated, as will be shown in Chapter XI. 

b) Planar fit.—The type of geometrie fit just described may also be in- 
terpreted as a planar fit. Each plane is defined by two of the reference axes, 
or by the end points of the two reference vector: 
metric fit is then indicated by the proximity of the vectors representing the 
variables in the common-factor space to such planes. 


A subset of variables may be well represented by vectors in a reference 
even though they do not form two dis- 
tors present a fan-shaped configuration 
lane is defined. The two reference axes 


: uch greater freedom than when the axes 
are restricted to the clusters of vectors. 


: The equivalence of the geometric and algebraic interpretation of a statis- 
tical variable will now be pointed out, As shown in Chapter II, a variable 
may be considered as a linear function of, say, m common factors. Geo- 


metrically, this means that the vector Tepresenting the variable lies in an 
m-space, which is defined by the m reference axes representing the factors. 
Thus if a vector lies in a p in terms of two reference 


; lane, it is describable 
vectors, and hence in the algebraic description of the variable only two 
common factors appear. Of course, other variables of the set might in- 
volve these or other factors. 

c) Hyperplanar fit.—In the two preceding standards one-spaces and two- 
Spaces were the bases for d. 


s etermining the adequacy of geometric fit. This 
idea can be extended to higher spaces. By hyperplanar fi 


s and the origin. Good geo- 


t in a space of m 
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dimensions is meant that each vector representing a variable in the com- 
mon-factor space lies in an (m — 1), or smaller, space. 

When a set of variables satisfies this standard, the complexity of any one 
of them is less than the total number of common factors. This does not ap- 
pear as a very stringent criterion at first sight, because it is satisfied if each 
variable is of complexity (m — 1) for m common factors. The strength of 
this standard, however, lies in the fact that the hyperplane is the largest 
permissible space containing each variable. In other words, it is hoped that 
there will always be smaller reference spaces which contain certain sub- 
groups of variables. Thus the complexities of the variables are reduced be- 
low (m — 1). In particular, a vector which lies on a reference axis is con- 
tained in that one-space, and the variable it represents has a complexity of 
one. Similarly, if a vector lies in a reference plane, the variable is of com- 
plexity two. This analogy of the smallest reference space in which a vector is 
contained and the complexity of the variable can be extended to any degree. 

It is evident that the two preceding geometrie standards may be con- 
sidered as special cases of hyperplanar fit. For, if a set of variables satisfies 
the criterion of linear fit, or planar fit, it certainly conforms to hyperplanar 
fit. The converse, of course, is not true generally. Hence the ideal to be 
aimed for is to reduce the hyperplanar fit to a geometric fit of as small a 
number of dimensions as possible, the limit being linear fit. 


6. GEOMETRIC FIT: POINT REPRESENTATION 
geometric representations of a set of variables 
ector representation, three standards 
pes of preferred 


In 3.7 two alternative 
were presented. By employing the v 
were immediately evident which will lead to as many ty i 
factor patterns. Now, by considering the point representation, another 
standard evolves. It will be recalled that in this representation there is one 
point for each of the N individuals, referred to a system of n reference axes— 
one for each variable. The points which are plotted in this n-space are con- 
tained in a common-factor space of only m dimensions.* The loci of the 
swarm of points of uniform frequency density are, more or less, concentric, 
similar, and similarly situated m-dimensional ellipsoids, being exactly so 
for a normally distributed population. It then seems natural to take the 


* According to Theorem 3.2, the N points, (zi 2a) - 5 £u), (£is 222) «++ 5 Zn), = « E 
(ais, Zay Zax), Whose coordinates are the elements in the columns of the matrix 
Ny Z0, ees y 


llzj:l|, are all contained in a linear m-space, where m is the rank of this matrix. Then the 

Argument of 3.8 indicates that the common-factor space (containing the N points) is of 

m dimensions, where m is the rank of the correlation matrix, with communalities in the 
, 


diagonal. 
tG. Udny Yule and M. G. Kendall 
Chap. XII. 


, An Introduction to the Theory of Statistics, 1937, 


94 FACTOR ANALYSIS 


principal axes of these ellipsoids as the fundamental reference axes. This 
standard, which is called ellipsoidal fit, leads to another preferred type of 
factor pattern.* 

From a statistical point of view several of the preferred patterns may fit 
a matrix of observed correlations equally well. The ultimate choice of type 
of factor pattern must then rest, in part, upon the nature of the variables 
and the utility of the solution in the particular field of investigation. 


5.3. Algebraic Solution of Any Symmetric Matriz 


Before describing the various preferred types of factor patterns, a solu- 
tion of special mathematical and historical interest will first be exhibited. 
This solution is obtainable by means of a general algebraie procedure for 
factoring any symmetric matrix, known as "completing the square." The 
method was applied specifically to a correlation matrix by MeMahonf be- 
fore 1923. In the early stages of factor analysis one writer{ developed the 
solid staircase method, while another§ also treated this type of solution 
under the title diagonal method. 


The n variables may be described in terms of 


n (or possibly fewer) new 
uncorrelated factors, 


by means of the above methods, in the following form: 


bd « 4 = au^, 
* 
2; = anF, + as; , 
(5.1) 23 = daly + aaf: + Asal"; , 


Zn = AmFy + GneFs + anaa +... “FP sul as. 


It is evident that a great many variation: 
are possible, since any one of the variabl 
one factor. A solution of the fo 
not definitive enough. 

The preferred t 
this chapter are we 


8 of this particular form of solution 
es may be selected as involving only 
rm (5.1) is therefore undesirable, since it is 


* The application of higher-dimensional ellipsoids in the field of factor analysis was 
proposed by Truman L. Kelley at a meeting of the Unitary Traits Committee in 1933. 
t James McMahon "Hyperspherical Goniometry; and I icati i 

, $ ts A) tion 

Theory for n Variables,” Biometrika, 1923. A H 

1 K. J. Holzin 


ger, Preliminary Report on 
No. 5. 


Spearman-Holzinger Unitary Trait Study, 


ST. L. Thurstone, The Vectors of Mind, 1935, pp. 78-81. 
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5.4. Uni-factor Pattern Ak 

The first of the preferred patterns to be discussed may be regarded as an 
ideal, since it is highly improbable that it will be attained with actual data. 
This, as well as all the other preferred types, is in agreement with the as- 
sumed composition of statistical variables. Another property which will 
hold true for all solutions in this chapter is that the factors are uncorrelated. 
The additional assumptions, which distinguish the particular pattern under 
discussion, are those of best geometric fit and minimum complexity. In the 
vector representation of variables the best geometric fit is attained when the 
projected vectors (into the common-factor space) of a group of variables lie 
close to a reference axis. When the vectors lie exactly on such axes, each 
variable then measures but a single factor. Of course, variables of distinct 
groups measure different factors. 

Such an ideal solution, which satisfies the standard of linear fit and hence 
minimum complexity, will be called a uni-factor pattern. This type of pat- 
tern will now be discussed in detail. Let there be variables and group 
factors (m < n) such that p: variables are expressible as linear functions of 
only the first factor Fi, p» other variables are expressible as linear functions 
of the second factor P», and so forth, to the remaining Pm variables which 
are expressible as linear functions of only the last factor P. The n variables 
may be subdivided into m groups Gi, Gs, ..- 5 Gm according to the factor 
involved in each variable, so that n = (pı + pe... + Pm). Then, if the 
variables are numbered consecutively from 1 to n, such a uni-factor pattern 


MEI 3 . -— 
(omitting unique factors) may be written as follows: 
a = au, 

Z: = agi , 


Zp, = apa, 


Ape , 


z = 
(5.2) pH 
Zyta = (p ipo » 
F. 
Zn-padi = Ü(n—p,,31)m m ; 
Zn = [ Re ee 


* A simplification in notation is employed in this and ll succeeding factor patterns. 
Since the linear expressions are approximations to observed variables, they should be 
denoted by z} in accordance with (2.4). Also when unique factors are omitted, En vari- 
ables may be considered as projected into the common-factor space and at e ie 
sented by z/. For simplicity, however, all primes are dropped, and it is understood tha 


any linear expression stands for an approximation to the indicated variable. 
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It is assumed in this pattern, and the two succeeding ones, that the coeffi- 
cients of the factors are positive. The contributions of the factors are evi- 
dently dependent on the number of variables in the subgroups which meas- 
ure the respective factors. When these subgroups are fairly equal, the con- 
tributions tend to be uniform. The uni-factor pattern may therefore be said 
to satisfy standard 4b of section 5.2. 

A simple schematic representation of a uni-factor pattern is given in 
Table 5.1. The vertical lines represent the coefficients of the respective fac- 
tors. Such a diagram then indicates clearly the mutually exclusive charac- 
ter of the subgroups of variables. 


TABLE 5.1 
UNI-FACTOR PATTERN SCHEME 


Variable Fy Fz Be aus Fa 


To distinguish betw 
theory notations* 
tions follow: 


a) eG, 


een variables of different groups, the standard set- 
can be adapted to the present needs. The primary defini- 


means e is a variable in the group G,. 


b) (zi; kG, s = 1,2,..., m) denotes the system of elements 2; 
for all values of j in groups Ga, where the range of s is in- 
dicated. The elements of the system are first designated, 
followed by a semicolon, and all the properties on the elements 

(5.3) are to the right of the semicolon, 

€ ZXGu-1,2,..., N) means the sum of all the elements in 
the system (zj; 4 = 1,2,..., N), The index j is fixed, the 
summation extending aver i. This sum is equivalent to the 


more conventional form > 2j. 


i=l 
«9g Hn « ee 
wien, emple, L. R. Wilcox, Modularity in the Theory of Lattices,” Annals of 
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It will be found that these definitions* aid in clarity and ultimate simplicity 


in describing much of the following theory. 
. Since the factors are always assumed to be in standard form, the correla- 
tion coefficient between any two variables jeGs and keG, is reproduced by 


means of pattern (5.2) as follows: 


" lx r 
Tk = Ilza; $12... N) 
a = d sas E e t= 1,2...) 


4 


= Gk F, F, 


When the two variables belong to different groups, the reproduced correla- 
tion becomes 


(5.5) ri, —0 (JG, , keG, , sz 0, 
since the factors are uncorrelated; whereas, if the variables are in the same 
group, 

(5.6) Tik = aiia (j,keG,) . 


(5.6), it is seen that & factor pattern of type (5.2) 
lations of the form shown in Figure 5.1. 


The plus signs in the small triangles represent positive correlations among 
variables within groups, and the zeros in the rectangles represent lack of 
correlation between variables in different groups. It is unlikely that such a 


picture of intercorrelations of variables would be obtained in practice, and 
termed "ideal" To the extent to 


hence a pattern of type (5.2) has been 
which a matrix of observed correlations approaches the above form, its solu- 
tion may be considered as approaching the uni-factor pattern. 

The only practical instance in which a solution conforms to the uni-factor 


consider a set of ten variables (az, 25 . .. , 


From formulas (5.5) and 
can only reproduce a set of corre 


* To illustrate the definitions given in (5.3), 
210), grouped as follows: 

G = (1, 2, 3, 4, 5: 

An instance of (a) is 7«G;, which means tha 


tem of elements (z;; jeG,, s = 1, 3) is the set of vari: í 
of N values of variable 4, which may þe taken as an example of (c), is 


Elzai = Bacca M- 
dividual values of the variables, 
lied in the development of (5.4). 


), G = (6,7,8), G, = (9, 10). 


t variable 7 is included in group G;. The sys- 
ables (z1, 22) Zs, Z 28, Z9, 210). The sum 


ten explicitly for the in 


T Pattern (5.2) could be writ r 
y, that is the form imp! 


as was done in eq. (2.5). Actuall 
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form is the classical ‘‘Two-factor” pattern.* This solution is — een 
factor" by Spearman because two types of factors general a : io vac 
are involved. From the point of view of complexity, however, t " : uim 
is a uni-factor one, since each variable involves just one common - o Long 
same one). To illustrate a '"Two-factor" pattern, suppose that four vs 


Fic. 5.1 


ables satisfy the tetrad conditions (4.6) for the existence of just one common 
factor. Then the pattern ma: 


y be written in the form: 


4 = ayo + alU, , 


22 = Anok’y + aUa " 
5.7 
en 23 = GsoF'g + aU; , 
Z4 = G4oFg y + aU. 


It will be noted that this 


pattern is a special case of (5.2), namely, for only 
one factor. The correlati 


ons of a set of variables leading to a ""Two-factor"' 


ose in one triangle of Figure 5.1, which satisfy the 
general conditions (4.13) for one general factor. 


5.5. Bi-factor Pattern 


orm indicated in Figure 5.1, and which 
is highly improbable of attainment. 


In a practical case, when a set of rele- 
* Charles Spearman, The Abilities of Man, 1927. 
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vant variables is employed to study the common attributes of a group of 
individuals, it is to be expected that the correlations between subgroups of 
such variables will be positive throughout rather than zero. Still higher 
correlations will usually be found for the variables within subgroups of the 
entire set. Such a matrix of correlations is indicated schematically in Fig- 
ure 5.2. The need for modifying the uni-factor type of pattern when dealing 


with observed data is thus apparent. 


1 ———— 


+ 
+ 


Fig. 5.2 


p factors postulated in the uni-factor 


Pattern, a general factor is also included, the positive correlations through- 
out the observed matrix can be accounted for. This modification of the uni- 
factor pattern is tantamount to substituting planar for linear geometric fit. 
When the vectors of a group of variables lie in a reference plane, each vari- 
able measures just two factors. If, furthermore, the vectors lie only in the 
reference planes formed by a general-factor (Fo) axis and one group-factor 
axis (none in the planes of two group factors), the configuration can be de- 
Scribed as a pencil of planes through the Fo axis.* A solution satisfying 
Standards 1, 2, 3, 4c, and planar fit, as here described, will be called a bi- 


factor pattern.} 


" re In ordinary space geometry 
h ine, i.e., all the planes linearly depe 
e present setting, however, & pencil of 
efer to all the planes through a line which are t 
that there are such planes in the pencil through the Fo axis. 

T The bi-factor pattern actually bec factor pattern when the factors are 


Correlated, as will be shown in 11.3. 


If, in addition to the distinct grou 


es refers to the totality of planes through 


dependent on two distinct planes through the line. In 
planes in & space of (m + 1) dimensions will 
mutually orthogonal. It is clearly seen 


a pencil of plan 


omes a uni- 
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The formal presentation of the bi-factor pattern will now be given. Let 
the n variables be expressible as linear functions of (m + 1) common factors 
as follows: every variable involves a general factor Fo, and, in addition, pi 
variables involve the first group factor /i, ps other variables involve the 
second group factor F», and so on to the remaining p, variables which in- 
volve the last group factor Fm. The n variables may be rearranged accord- 
ing to the groups Gi, Gz, ... , Gn, and numbered consecutively from 1 to n. 
Then, employing the notation of the preceding section, the bi-factor pattern, 
omitting unique factors, may be written in the form 


Z, = aol) + auFy , 
22 = Fo + an^, 


Zp, = Apolo + ay Fi, 


Zp +1 = (p ijo + ama? , 
(5.8) 
Zpkp, = Up, +p,)0lo + (p pps, 
Zn-omtl = AUn—p,+1)0!"9 F Qn pm ttyml mn , 
Zn = Gus F inl ns 


A schematic plan of the bi-factor pattern, similar to th 


i : at given for the 
uni-factor pattern, is presented in Table 5.2. It is apparer 


at from this dia- 
TABLE 5.2 
BI-FACTOR PATTERN SCHEME 


Variable Fo Fy Fa Fa F, 
e Fm 


PREFERRED TYPES OF ORTHOGONAL SOLUTIONS 101 


gram that each of the group factors overlaps with the general factor but 


not with any of the others. 
The correlation between any two variables jeG, and keG, may be repro- 


duced by means of a bi-factor pattern as follows: 


Tj = y Zt H i=l, V AE N), 
(5.9) = d E([a;oFo; + ajs Faila roF oi + auFid;27212..., N), 


= a5sü4o + aj ur pF, F Gi AT F,F, T asa FEL a 


= Ajoko + aj T F,Fe o 


rom the fact that the general factor is un- 


Where the last equality follows f actor 
If the two variables are in different 


Correlated with the group factors. 
groups, then 
(5.10) T, = Ajolo (jeG, , keG,, s #1), 


produced correlation becomes 


but, if they are in the same group, their re 
(j,keGs) . 


(5.11) r[, = Ajodxo F Gin 
that the relatively small correlations 


between variables of different groups are produced through the general fac- 
tor Fo. These correlations appear in the rectangles of a correlation matrix 
as in Figure 5.2. The higher correlations between variables within a group 
are reproduced by formula (5.1 1), which involves the general and one group 
factor. Such correlations are represented in the triangles of Figure 5.2. 
The form of the bi-factor pattern was conceived* as a direct eaten of 
Spearman’s “Two-factor” solution in the case of sets of variables noa do 
not meet the assumptions underlying the latter solution. For a specialized 


i actor, may be 
ie i "Two- ? pattern, i.e., one general factor, m 
af volaba tha Magnis: With a complex set of variables, 


Sufficient to reproduce the correlations. ? 
produce the 

however, the criteria for a single general factor may uera em » 

Eroup factors may be required. The bi-factor solution makes a 


oth general and group factors. 


From equation (5.10) it is evident 


5.6. Multi r 
" ple-Factor Pattern - 
An alternative solution to the bi-factor pattern which accounts for a table 


i i neral 
of essentially positive correlations, but which does not involve a gene 


-Holzinger Unitary Trait Study, 
"E oF r ;minary Report on Spearman d 
No. 5, Fee e is dà bi-factor form was proposed by Frances Swineford 


during the preparation of Report 5 


j 
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factor, will be called a multiple-factor pattern.* In this solution the next 
stage of geometric fit, viz., hyperplanar, is employed. Analytically, this 
means that a number of "overlapping" group factors must be postulated in 
order to account for the generally positive correlations among a set of vari- 
ables, as exhibited in Figure 5.2. These factors are called "overlapping" to 
distinguish them from the distinct group factors which are employed in the 
pattern (5.2). The term "overlapping" indicates that several factors appear 
in the description of a variable. Thus, in the bi-factor pattern the group 
factors overlap with the general factor, whereas in the multiple-factor pat- 
tern the group factors overlap among themselves. Overlapping thus implies 
a complexity of two or more. By sacrificing the general factor of a bi-factor 
pattern, a higher complexity than two will usually be required in a multiple- 
factor pattern. It is for this reason that the prefix "multiple" is employed. 

À multiple-factor pattern satisfies standards 1, 2, 3, 4b, and 5c. The 
Standards of parsimony and hyperplanar fit may be stated in analytical 
terms. In conformity with these Standards, it is desirable that the com- 
plexities of the variables in a multiple-factor pattern be as small as possible; 
and, in spite of the necessary overlapping of factors, it is also desirable that 
there be as many zero coefficients in the columns as possible.t If these con- 
ditions are satisfied, then the factors can be identified by means of the 
groups of variables with appreciable coefficients. 

For a given table of correlations there should be a unique multiple-factor 
pattern with a large number of zero coefficients. However, the particular 
configuration of zeros and overlap of group factors will usually vary for dif- 
ferent sets of variables. Hence no general mathematical formulation of the 
multiple-factor pattern is possible apart from a given matrix of correlations. 
Therefore, no schematic plan for this type of solution is presented. An illus- 


* Although a multiple-factor pattern is here formulated as one of the preferred types, 
tual practice it must be derived b 


pri y rotation from some preliminary solution. One 
um m pattern, known as the centroid pattern, will be described in detail in 
ap. VIII. 


s t Such a configuration, with correlated or uncorrelated factors, has been called & 
n e by Thurstone (op. cit., Chap. VI) when the following conditions are 
satisfied: 


1. Each row of the factor structure V (see 2.4) should have at least one zero 
2. Each column of V Should have at least m zeros (m being the total number of com- 
mon factors) 


3. For every pair of columns of V there should be at least m variables whose entries 
vanish in one column but not in the other 
With special reference to psychology, 


ferer : Thurstone has called the factors determined by 
hyperplanar fit primary abilities.” 


1 
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tration of a possible form of a multiple-factor pattern, however, is given for 
a special case of twelve variables and four factors: 


z = anā, d aeFs + asks, 
Zo = anF + ayFs , 
z; = aaf, + asFs dass, 
Z4 = agli + aks, t 
2a = ase, + assF3 + ass, 
Zs = aaF. + dels, 

(5.12) z = aF + anFs + anFs, 
Zs = agli + asF2, 
Z = aF + ass + asfi, 
Zi = aiii + aio,3F3, 
Zu = anii + an Fs, 
Z = y2,2F2 + ax,3Fs + ay, ls. 


As in the case of the uni-factor pattern, the contributions of thése factors 
are approximately level. From such a pattern, it is obvious that the correla- 
tions are reproduced by the general formula (2.22). Every pair of variables 
is linked by overlapping group factors and thus yields a positive correlation. 
A given matrix of correlations, of the form in Figure 5.2, can thus be re- 
produced by a multiple-factor pattern. M . 

Although the three preferred types of patterns are formally distinct, with 
actual data the resulting solution may be a combination of them to a cer- 
tain extent. Thus a solution may be of the bi-factor form but have some 
overlap of group factors and, to this extent, resemble a multiple-factor solu- 
tion. Likewise a pattern of the multiple-factor type may have a factor ap- 
Pearing essentially in all variables and thus resemble the bi-factor form. 
Either the bi-factor or the multiple-factor types may involve several vari- 
ables which approximate unit complexity and thus partially approach the 


ideal uni-factor form. 


5.7. Principal-Factor Pattern 

The preferred types of factor patterns exhibited thus far are of a special 
kind, that is, they require that all factor coefficients be positive, and the 
multiple-factor pattern furthermore precludes a general factor. if a matrix 
of correlations is observed to be in the form of Figure 5.1, there is no doubt 
but that a uni-factor solution is most meaningful. On the’other hand, when 
2 table of correlations appears as in Figure 5.2, there is no assurance, from 
the statistics alone, that a bi-factor or multiple-factor solution will afford 
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the most useful interpretation of the variables. The nature of the data may 
then aid in the selection of a particular form of representation. It is con- 
ceivable that certain sets of variables may be better interpreted in terms of 
factors, the coefficients of which are not restricted to be positive. 

A factor for which several of the coefficients are positive and others are 
negative will be called a bipolar factor.* Such a faetor may appear in all 
variables of a set or only in a subgroup of them. A bipolar factor is not es- 
sentially different from any other but is merely one for which several of the 
variables have significant negative projections. Such variables may be re- 
garded as measuring the negative aspect of the usual type of factor. Thus, 
if à number of variables identified with "fear" are represented by positive 
projections, variables with negative projections might be interpreted as 
measuring “courage.” It would appear simpler, however, to regard the fac- 
tor merely as “fear,” and the opposing set of variables as measures of ‘‘nega- 
tive fear." Of course, the signs of all the coefficients of the factor may be 
changed without altering the adequacy of the solution. Such reversal in the 
foregoing example would lead to the interpretation of the factor as ‘cour- 
age," and the subgroup of variables with negative coefficients would be re- 
garded as measuring “negative courage." In the illustrations of the text. a 
single name for a bipolar factor will be employed. This is consistent with 
representing any factor by a single continuum, 

The preferred type of pattern no 
which are present in all variabl 
factors, there is also a general f 
Such a solution which satisfies s 


w to be discussed involves bipolar factors 
es of the set. In addition to these bipolar 
actor with positive coefficients throughout. 
tandards 1, 2a, 3, 4a, and 6 is called a prin- 


* This term was intr 
Traits,” Character and 
more general sense. 


oduced by Cyril Burt in 


“The Factorial Analysis of Emotional 
Personality. In the prese. 


nt text, however, the term is used in & 

+ Harold Hotelling devised a solution of this fi 
or the reliability of each variable instead of 
al Variables into P. 


orm but factored the total unit variance 
the (estimate of) communality (cf. “Analysis 


of a Complex of Statistic rincipal Components," Journal of Educational 


Psychology, 1933). 
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A principal-factor pattern, in which the unique factors have again been 
omitted, may be exhibited as follows: 


z = any taps + Fa +... + ass, 
Zo = anfi + agf + assF3 +... + damm , 
(5.13) zy = agi + agel’s + assFs +... + ass, 


= aml, + anel’s + Qnsl’3 +... + hl s 2 

In such a pattern the complexity of each variable is equal to the total num- 
ber of common factors. The first factor is an ordinary general factor whose 
coefficients aj (j = 1,2, . . . , n) are all positive when the solution is based 
upon a table of positive correlations. On the other hand, approximately 
half the coefficients of each of the remaining factors are negative, that is, 
Fa, F3,..., Fm are bipolar factors. 

Obviously a pattern of type (5.13) can reproduce a general table of corre- 
lations like those exhibited in Figure 5.2. The solution is thus perfectly 
satisfactory from a statistical point of view. Although a principal-factor 
pattern may be more complex than the other preferred types, it may some- 
times furnish a more convenient representation of a particular set of vari- 
ables. . 

Since the uni-factor, bi-factor, and multiple-factor patterns involve only 
Positive coefficients, they can reproduce a table of positive correlations only. 
In actual analysis, therefore, these types are adapted only to sets of vari- 
ables which are positively correlated or to sets which can be put in that 
form. If there are a few insignificant negative correlations, the preceding 
types of solutions can be obtained. If a variable has significant negative 
correlations with all the rest, its scale may be reversed, thereby changing 
the signs of the correlations. One such example is that in which a measure 
of stupidity is changed to that of intelligence by changing the sign of > 
scale. By this procedure of “reflection of variables” a matrix of essentially 
Positive correlations might be obtained. C7 , 

1f, after such Pina n significant negative and positive a ge 
appear for some of the variables, the first three preferred types would no 
be applicable. Since the principal-factor pattern is not eg to agen 
Coefficients, it can reproduce negative as well as positive mea meg i : : 
type of solution may then be applied to any matrix of ane sane i 
ample of a principal-factor solution based upon eme ct ne ee 
ations is given in 7.5. In this solution all factors, 


of the bipolar form. 
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5.8. Summary of Preferred Patterns 


It will be noted that the names “uni-factor,” “bi-factor,” and ‘“multiple- 
factor" indicate the complexities of the variables in the respective patterns. 
Thus in the uni-factor pattern each variable is of complexity one, while in 
the bi-factor pattern each variable is of complexity two. In the multiple- 
factor pattern the complexities of the variables may differ greatly, but no 
variable involves all the factors. As already indicated, the principal-factor 
pattern takes its name from the property of the reference system rather 
than from the complexities of the variables, which are generally equal to 
m for m common factors. In recapitulation the various assumptions 


and properties of the four preferred types of factor patterns are given in 
Table 5.3. 


TABLE 5.3 
ASSUMPTIONS AND PROPERTIES OF PREFERRED PATTERNS 
Type of Assumptions Number Complexity Distinguishing 
Pattern (Enumerated in 5.2) of omen ot Bact Characteristics 
Uni-factor........... 1, 2a, 2b, 3, 4b, 5a m +1 Distinet group 
Bit factors 
PPG OLOR cpi site 1, 2a, 2b, 3, 4c, 5b | mor m+1 2 One general plus 
k zroup factors 
Multiple-factor....... 1, 2a, 2b, 3, 4b, 5c m «m OVerldbping 
cs roup factors 
Principal-factor....... 1, 2a, 3, 4a, 6 m m One general plus 
bipolar factors 


From the above discussion it is apparent that the uni 
the ideal sought in all factor analysis. The remaining pre 
various standards for acceptable solutions, but these crite 
the complete basis for a choice of form. This choice will 
the nature of the variables and the theories or laws in a 
application. Thus, if an investigator considers the variables to be of the 

bipolar" type, the principle-factor form would be appropriate. A simple 
example of such a set of variables, in the field of biology, is dise by stand- 
ing height, sitting height, and length of forearm as oppased to weight, chest 
girth, and chest width. If a general factor were denied by a theory in a par- 
ticular field, then the multiple-factor type would be consistent; but, if the 
general factor were accepted in such a theory, then the bi-factor form would 
be suitable. Such contrasting theories now exist in the field of psychological 
abilities, but a theory including a general factor is commonly accepted for 
physical traits in biology. According to another theory, an analysis might 
be desired in which the factors appear in the order of their importance. This 


-factor pattern is 
ferred types meet 
ria do not furnish 
also depend upon 
particular field of 


A 
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type of importance is indicated by the decreasing contributions of successive 
factors of the principal-factor solution. It is thus evident that general scien- 
tific, as well as statistical, standards in any field of investigation must be 
employed in the selection of a preferred pattern. 

It will be shown in Part III that, once estimates of communalities have 
been made for a given solution, the latter may be converted into any other 
form desired by a suitable transformation, and all such solutions will fit the 
Correlations equally well. Preference for a given form must thus depend 
upon statistical standards, such as those discussed in this chapter as well as 
criteria apart from the statistics. Examples of such criteria are Spearman’s 
Psychological theory of a general factor generalized to the bi-factor form 
and Thurstone’s contrasting theory of primary abilities in which the general 
factor is not included. 

Tn psychology, or any other field, 


Weng y and show that it is compatib’ 
Possible, however, that another investigator may postulate an alternative 


theory also compatible with such evidence. The mathematical expressions 
(the patterns) of these theories may be formally different because the stand- 
ards employed in the two cases would not be the same. Inasmuch as both 
theories are consistent with empirical evidence, neither should be used as a 
asis for judging the appropriateness, oT inappropriateness, of i other. 
his type of conditioned thinking has been used in factor analysis, ied 
eading to much confusion as to the ultimate form of solution to be adopte : 
In the application of factor analysis to psychology it has been argue 


that Were it is inconsistent. with another 
one form n is incorrect because T ) 
mae or form of solution has been rejected 


st battery, while 


an investigator is free to postulate a 
le with experimental evidence. It is 


asker is invariant for the battery. It 
solutions are dependent on the partic 
“tgument for choice of solution canno 


Invaria 
nee. ultimately selected, depends upon the 


The for lution : 
following sien bep irem of individuals meum hadaga i ` 
Variables and all their intercorrelations; (c) phai a ihe particular 

Ose in the present chapter; and (d) outside criteria veis is obviously 
field of investigation. Like all other statistics, cone p ieri 

“Pendent upon the particular sample of individue 2 arent that factor 
may be made by conventional methods. It is en nt dant upon all the 
smalysis is a correlational analysis and, therefore, = pea : are added to a 
Variables producing the correlations. If additional variable 
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given set, the table of correlations, as a whole, will usually change. — 
much as any factor solution is based upon all the intercorrelations, it wi 
vary as the set of variables is changed. Practically, of course, the ^x. 
of only a few variables may not change the resulting solution appreciab y. . 
The dependence of a factor solution upon the groups and the — 
analogous to the dependence of a multiple regression equation upon t » 
sample and the particular variables used for estimation. Statistical stand- 
ards and criteria from a given field are introduced so as to obtain a determi- 
nate solution in harmony with a postulated theory. 2. 
The above considerations should be clearly evident if factor analysis is 
viewed as a statistical method yielding solutions which are convertible from 
one to another. Because of their statistical nature, factor solutions will, in 
general, not be invariant, and because of their convertibility a preference of 
form can be made only upon employing appropriate outside criteria. In fac- 
tor analysis, as in all empirical sciences, several equally satisfactory lan 
may be usefully employed, although they may be formally quite different. 
In the field of psychology attempts have been made to formulate invari- 
ant solutions. The theory underlying such invariant factors involves the 
arbitrary specification of the four aspects upon which factor analysis de- 
pends. Thus by fixing the population and the set of variables, and agreeing 
upon the form of solution, a fundamental set of factors may be obtained. 
Then all other variables in the given field may be expressed in terms of these 
factors. 
After the methods of solution for various types of factor patterns have 
been presented, the above points will be discussed in connection with the 
choice of form of solution for practical problems (see Chap. XIII). The prob- 


lem of invariance, or stability of solutions, will again be considered and il- 
lustrated with actual data, 


* A somewhat analogous situation ari 
dynamics. A clear discussion of this prol 


ynamics, p. 2). —-— 
cording to these two approaches are called the “Eulerian 
of the hydrokinetic equations. In like manner, it has been 


to label a particular form of factor solution after the man 
8 development. 


and the "Lagrangian" forms 
the custom in factor analysis 
who has done most toward it 


PART II 
DIRECT ORTHOGONAL SOLUTIONS 


CHAPTER VI 
THE BI-FACTOR SOLUTION AND SAMPLING FORMULAS 


6.1. Introduction 


The foundations and formal groundwork for factor analysis have been 
laid in the preceding chapters. Now the analytical methods that may be 
employed in the solution of a practical problem will be developed in detail. 
In the present chapter the method of obtaining a bi-factor pattern for a set 
of variables will be discussed. 

To pave the way for a bi-factor analysis, the complete solution of Spear- 
man's “Two-factor” pattern is given; first, in the classical manner in 6.2, 
and then by a revised method. The assumptions underlying either method 
of solution are examined in detail so that the validity of a '"Two-factor" 
pattern can be clearly judged. In 6.4 the solution for a bi-factor pattern is 
developed. The plan of Chapter V is adopted, and the methods of 6.2 and 
6.3 are adjusted, to this more complex situation. When the simple plan (5.8) 
cannot be met by a particular set of variables, certain modifications must 
be made. Such revisions and verification of the simple bi-factor pattern are 
considered in 6.9. 

To aid in the judgment of the adequacy of any factor solution, certain 
sampling error formulas are necessary. The first of these—for the standard 
error of a general-factor coefficient—is developed in 6.5. Then a formula for 
the standard error of a residual is worked out for the case of any number of 
factors. This formula, however, involves the standard errors of the pattern 
coefficients. Additional sampling error formulas, based upon further ap- 
proximations, are presented in 6.7. Then, to facilitate the computation in 
an actual problem, tables of their values are given in Appendix H. f 

To illustrate the analytical methods of this chapter several numerical 
examples are employed. A set of five physical variables is used to clarify 
some of the ideas in connection with the sampling error formulas. The bat- 
tery of twenty-four psychological tests, which has been introduced in Chap- 
ter II, is employed in 6.10, where the bi-factor solution is exhibited. The 
complete outline for the computation of this pattern is given in Appendix C. 
It is believed that the procedure for calculation will be useful for reference 
purposes after the reader has become acquainted with the logical basis of 
the method, and it was therefore separated from the text and put in the 
2ppendix. i 
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6.2. Spearman’s ''T'wo-Factor" Solution 


According to Spearman's fundamental theorem,* the necessary and suffi- 
cient conditions for a set of n variables to be describable in terms of just one 
general factor and n unique factors are the vanishing of all tetrads.t The 
relationships between the tetrad conditions and the more parsimonious 
triads have been discussed in 4.3. The “Two-factor” type of pattern may 
be written as follows: 


(6.1) Zi = ajoFs + a,U; (j —1,9... 24), 


where Fois the general factor and the U; are the n unique factors, and again, 
as in the case of (2.4), the prime on zi has been dropped for simplicity. When 
the correlations rj; (j, k = 1,2,..., n) satisfy the tetrad conditions, or the 
equivalent conditions (4.13), the pattern (6.1) may be assumed and the 
coefficients ajo and a; have to be determined. In the present section formu- 
las will be developed for the computation of the ajo under the assumption 
that the residuals 


> , 
Vik = Tjk — Tik 


vanish. Then, in the following section, formulas for the ajo Will be obtained 
merely on the assumption that the conditions (4.13) are satisfied statisti- 
cally, i.e., within errors of sampling. 


The correlations reproduced from the pattern (6.1) are given by 
Tik = Qjodyo, 


and, under the assumption that the residuals vanish, the observed correla- 
tions may be written 


(6.2) Tik = Tjk = Qjoügo. 


In the remainder of this section the reproduced correlations will be replaced 
by observed correlations. Upon multiplying equation (6.2) by the square 
of the general-factor coefficient for any variable z,, this equation becomes 


(6.3) 


Gor; = A2yG joo = (Ae00j0) (eodxo) = Tejfek y 


and, summing over the correlations, there results 


n n 
(6.4) a, r= effet ie is E 
>> s : Jj kse 


* Spearman's theorem was deduced s, 
cable to any set of statistical variables. 


T Formulas for the sampling errors of tetrad differences are developed by Spearman 


and Holzinger, “Note on the Sampling Error of Tetrad Differences," British Journal of 
Psychology, 1925. 


pecifically for psychological tests, but it is appli- 
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As in 2.7, the symbol Hk rj stands for the sum of all the correlations 
j<k=1 

Tir, Where j and k each range over the variables 1, 2,..., n but subject 

to the restriction that j is always less than k. In a matrix of correlations, 

this sum is merely the total of all the entries above (or below) the principal 

diagonal. 

Since there is only one common factor, the coefficient of this factor for 
any variable is merely the square root of the communality of the variable. 
Hence formula (6.4) may be written explicitly for the square of the coeffi- 
cient, or the communality, of any variable za as follows: 


MCTTIENEURIILITETO 
6. 2 og Beeri J, 145 1M ds : . 
SEN n h Brn; h k= l, 2.o h jik e j <k) 


It will be observed that the diagonal elements of the correlation matrix do 
not enter into formula (6.5). In fact, the formula yields values of the com- 
munalities, which theoretically are the diagonal elements preserving the 
unit rank of the matrix. When the conditions (4.13), which are applied to 


observed correlations, are satisfied, & single factor is postulated. The com- 


puted diagonal elements must also satisfy these conditions in order that 


the rank of the complete correlation matrix be unity. . 
It may be illuminating to write out formula (6.5) in detail for a particular 


case. Thus for n variables, the square of the first coefficient is 


a us 
aio = 

(6.6) Tigris + Maria + s riin + riara + e Tian +... Tinin 
Tas F ra Heset Tm + ra E.F fs Fo t Ta 


For purposes of computation, formula (6.5) may be expressed as follows :* 


` »2 i 208 C is e 


j= j=1 
(6.7) €— e dee 
2( 3 i Ee 
j<k=1 j=1 


) would be written as follows: 


nn je 
+n, j #6) 


* In the set-theory notations formula (6.7 
PEE Bers Fa 
on. j SR) — Sta T= bee 


2 [2ra J = 
2Iz(s j, k 


PET: 


Go = ho Essa 
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The adaptability of this formula to machine calculation will be clear from 
the following restatements of the terms in the formula. Let R be the matrix 
of correlations with the elements in the principal diagonal omitted, then 


n 


> Tej is the sum of the correlations in column e of R, 
j=1 


n 
> 72; is the sum of squares of the correlations in column e, 
j=l 

n 


>. 7j, is the sum of all the correlations below the diagonal. 
j<k=1 


After the communalities, and hence the coefficients of the general factor, 
have been obtained, the unique variances can be determined. As pointed 
out in 2.3, the uniqueness for variable e is given by 


aj21-—Hh. 


If, in addition, the reliability coefficients of the variables are known, the 


uniqueness may be split into unreliability and specificity by means of formu- 
las (2.11) and (2.12). 


6.3. Computation of a “Two-Factor” Pattern by the Method of Triads 


The method of analysis that will now be described may be employed in 
place of the procedure of the last section. If the conditions for one common 
factor are satisfied by the correlations among n variables, the pattern (6.1) 
may again be assumed. It will become evident that the conditions to be 
checked for a general factor are actually included in the computation of the 
coefficients of that factor and therefore involve no superfluous calculations. 
If one suspeets that a general factor will account for the intercorrelations, he 
may proceed to the calculation of a "T wo-factor" pattern, and in the course 
of this analysis he may determine the adequacy of such a hypothesis. 

The grouping of variables by means of B-coefficients may lead the in- 
vestigator to the hypothesis of just one factor. When the correlations among 
n variables indicate the sufficiency of just one common factor, then the 
communality of any variable z, is given by any one of the following » — 


n—1N,. B 
2 ) triads: 


"T" T e is fixed 
i lj = = Gp hie dy 2, eae ms 


jd pke 


4 


+ 
L: 
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Each triad is subject to a small sampling error, so that the » numbers, 
tie, lia... ls (e omitted), are generally distinct. The most representa- 
tive number A2, in the least-square sense, is that which makes the quantity 
E(t — A225 j, k = 1, 2.2 ISA, j, k = e) a minimum. The mini- 
mum valuc is given by the arithmetic mean of the set of numbers tjx, that is, 


h == X(t; j, k = 1, 2,..., E RA?) 


(6.9) 


NL 


D (Aikawa on j<k, jkse). 


Tik 


| 


In determining the communality of a variable, therefore, the mean of all 


possible triads is used. Since there is only one common factor, formula (6.9) 
also gives the square of the coefficient of that factor for any variable ze. 
The basie assumption which validates formula (6.9), or formula (6.7), is 
that the set of variables can be described in terms of just one common fac- 
tor. In addition, formula (6.7) was deduced under the assumption that the 
residuals were zero. On the other hand, the conditions for just one com- 
mon factor are the equality of all the triads for any variable; so that, strictly 
speaking, all the triads in (6.9) should be equal. The essential difference 
between the method of 6.2 and this section is that in the former case the 
additional assumption is made explicitly in the derivation of the formulas, 
Whereas in the latter case no assumptions are made aside from the basic one. 
Of course, the conditions for one general factor are expected to be met only 
within errors of sampling. The method of this section involves the compu- 
tation of individual triads, which may be inspected for statistical equality. 
In the following sections sampling error formulas will be developed which 
may be of use in this connection. . 
There is one further assumption which must be made for.the practical 
problem. When the solution for the communality h2 is made by either for- 


mula (6.7) or (6.9), it is tacitly assumed that 


e is fixed 
Tejfek 5d 1,2; mls 
6. Wak aj js 125-3 
dn Tj j<k, ike 


Furthermore, for any statistical variable h? € Tez, SO that 


Teifek ac ye 
(6.11) ELLE 
where r,s is the reliability coefficient of variable ze These aomp ke 
usually be satisfied in practical problems, but if a particular mi exceeds 
unity it should be dropped from the evaluation of the communality. 
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6.4. The Bi-factor Method 


The *"Two-factor" pattern is very limited in its application because it 
implies that the rank of the matrix of correlations is unity. Such a simple 
relationship among the correlations is not ordinarily to be expected for a 
large set of variables. The method of analysis as given in 6.2 and 6.3 is very 
useful, however, in the more general bi-factor form of solution. A bi-factor 
pattern of the type (5.8), or one with slight modifications, can be obtained 
for any matrix of positive correlations by applying the preceding methods. 

The procedure is to select appropriate variables out of the total set, which 
taken alone have a matrix of correlations of rank one, and then to apply the 
analysis for variables involving only one common factor. This procedure is 
justified by Theorem 3.5, which states that the dimension of the common- 
factor space is equal to the rank of the correlation matrix. The common- 
factor space in the new situation refers, of course, only to the space contain- 
ing the specified variables. Now these variables are selected from the pat- 
tern plan to involve only one common factor, and so the rank of their ma- 
trix of correlations must be one. 

The complete development of a bi-factor pattern will now be given. Sup- 
pose a set of n variables has been shown to be divisible into m groups G, 
(s = 1,2,..., m) by the method of B-coefficients. If the m groups involve 
Dy, 3... , Pm Variables, respectively, the bi-factor type of solution can be 
written as in (5.8). For convenience the pattern* is repeated here: 


z = as + anky 
Z2 = dsoF o + an, 
Zp = ap, Fo + ap Fi 

Zp = (p, +1)0F0 + a )2F2 

(6.12) T— P 
Zp+r, = Up, +p)0F'0 F aip, +p)2F2 
Zn-pm+1 = A(n-pm+1)0F0 O M 

Zn = anolo d su. 


The problem now is to determine values for the coefficients in pattern (6.12) 
which will reproduce the original correlations, within errors of sampling. 


* See n. *, p. 95. 


THE BI-FACTOR SOLUTION AND SAMPLING FORMULAS 117 


The computation of the coefficients falls into two parts: first, the general- 
factor coefficients, then, the group-factor coefficients. 

Before developing the methods of obtaining the factor coefficients, the 
geometric interpretation of the bi-factor plan will be briefly reviewed. The 
common-factor space which is used to describe the n variables in (6.12) is 
clearly of (m 4- 1) dimensions, there being that many factors or axes. It 
may be possible, perhaps, to describe the variables in a space of only m di- 
mensions, but the complexity of each variable would then be greater. There 
is a great deal of geometrical elegance and statistical simplicity in describing 
the variables in the form (6.12). Although the total common-factor space 
is of (m + 1) dimensions, each vector representing a variable lies in a plane, 
that is, each variable measures only the general and one group factor. The 
total configuration is described as a pencil of planes through the Fo axis.* 
When no variable measures more than one group factor, there are no points 
in the planes formed by pairs of group-factor axes.] This geometric inter- 
pretation of the bi-factor configuration gave the clue for further analysis 
into correlated factors, which will be presented in Chapter XI. 

The general-factor coefficients of a bi-factor pattern can be obtained by 
the method of 6.2, applied to appropriate subsets of variables. Any one of 
the triples (e, j, k) of the system ((e, j, k); eG, JeGs, keGs, U 75 8 n i 
u, s, t = 1,2,..., m) involves only one common factor. Hence, assuming 
that the residuals vanish and replacing correlations from the pattern by 
observed correlations, there results 

Tjik = Qjolko (jeGa, keG 8 # t) 
according to (5.10). Then, multiplying both sides of this equation by ao, 
and reapplying this equation, the following expression arises: 


(6.13 als = Aojo = (deo jo) (100.0) = Tele. 
e0' j u 


2, as computed from the correlations with 
ain a more reliable evaluation of any 
des of (6.13) for all valuesf of j and k 


Formula (6.13) gives the value of a 
partieular variables j and k. To obt 
general-factor coefficient, sum both si 


* Cf. 5.5. 
1 An exception occurs when a variable measures & single group factor and not even 


the general factor. In this case the vector representing the variable actually lies on the 
group-factor reference axis, and hence in the plane of this axis and each of the other group- 


factor axes. 


t If j and k are merel. 
all groups, each correlati 
j and k are permitted to range ove 
m, but under the condition s < t. 


y restricted to be in different groups, and the variables range over 
on would appear twice, since Tjk = Tki TO avoid this, the indices 
r all groups, namely, jeG, and keG, for s, t = 1, asy. 
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which, together with e, preserve the property of involving only one com- 
mon factor. For any eeG, it follows that 


Zara jeGs, keGa, 8 «0, 8,025, 8,6: 1, 2,....,-m) 
2 = 
(6.14) af Z(ris; jeGu, kG, s « t, 5,025 BW 8, f= 1,2,..., f) 


This formula gives the square of the general-factor coefficient for any vari- 
able z.. 


To illustrate the application of formula (6.14) and to clarify the notation, 
a concrete algebraic example will be considered. Let there be fourteen vari- 
ables which, according to the B-coefficient technique, group as follows: 


6; = (1, 2,3), Gs = (8, 9, 10, 11), 
G: = (4, 5, 6, 7), G, = (12, 18, 14). 
Then the bi-factor plan may be formulated as in Table 6.1. The general- 


TABLE 6.1 


BI-FACTOR PATTERN PLAN FOR FOURTEEN 
HYPOTHETICAL VARIABLES 


Variable Fo Ln F: Fi Fi 


Mina 
aisa 
Ou 


factor coefficients can be computed by means of formula (6.14). For ex- 
ample, the first coefficient is given by 

a = Era 8, keGy s < 1, st = 2,3, 4) N 

A Eliri eG., keGi s < t, 8,6 = 2,3, 4). ^ D^ 


where N and D have been used to denote the nu 
respectively. As it stands, this formula involve: 
tions in the denominator and the sum of the sa: 


merator and denominator, 
s the sum of forty correla- 
me number of paired prod- 


THE BI-FACTOR SOLUTION AND SAMPLING FORMULAS 119 


ucts of correlations in the numerator. The denominator may be written in 
the more conventional (and longer) form, namely, 


7 


7 14 11 14 
2293930293932 


j-4 k-8 j-8 k-12 


After some factoring (algebraically), the numerator may be written as 
follows: 
N = (rij; JeGa) E (rin; keG,, s = 3, 4) + Z(n5 JeGs)Z (rae; EeG) , 


or more explicitly in the form 


N = (ru + ris + rie + Tis) (ris + Tio + Thio + Tun + Tuas + Tas 
a 71,14) T (ris + rig + 71,10 + 71,11) (71,12 + tis + 71,14). 


From the values of N and D, the coefficient dio is readily obtained. 
The correlations which are involved in the calculation of ajo may be repre- 
Sented schematically as in Figure 6.1. The numerator consists of the sum 


8 9 14 


1 12 13 


10 


c o|-1 o om [co m = 


mi pa 
- Oo 


Fra. 6.1 
ions appearing in the first column, while 
enominator are merely indicated by the 


of appropriate products of correlat 
the correlations entering into the d 
blocks showing the locations of the v: 
Work for routine calculation of a num! 
dix C 


ariables j and k. The arrangement of 
erical problem is described in Appen- 
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An important extension of formula (6.14) should be noted. If the pattern 
plan of a set of variables is of the bi-factor form, but includes a number of 
variables which measure only the general and no group factors, additional 
terms can be included in formula (6.14). For any two such variables, to- 
gether with any other variable e, will involve only one common factor. The 
summations in (6.14) should extend to all such variables j, k; the only re- 
Striction being j « k so that no correlation should be used more than once. 
There seems to be no reason for unnecessarily complicating formula (6.14) 
to cope with this case, it being clear how the formula is extended. 

After the general-factor coefficients are computed there remains the prob- 
lem of determining the group-factor coefficients. The first step is to obtain 
the residual correlations with the general factor removed, or the general- 
factor residuals as they will be called hereafter. These are defined by 


(6.15) Fik = Tik — Gjodko (j,k =1,2,..., n). 


In general, the table of these residual correlations will be of the form shown 

in Figure 5.1, the values in the rectangles being approximately equal to 

zero. The standard error of a residual correlation will be developed in a 

ae section, so that the significance of any entry in this table can be veri- 
ed. 

In the residual-factor space, the n variables can be described by a uni- 
factor pattern, i.e., the bi-factor pattern may be considered as a uni-factor 
pattern with a general factor superimposed. The residual correlations (6.15) 
for each group of variables, taken alone, should have a matrix of rank one 
and hence measure only one common factor. Either of the methods of 6.2 
or 6.3 can now be used for the calculation of the group-factor coefficients. 
Since there will usually be a relatively small number of variables in each 
group, it seems more advisable to use the method of triads of 6.3. By this 
procedure the group-factor coefficient for any variable eeG, is given by 


NE: ha. 
(6.16) a, = ‘on Z =? jkG, j< k, j, k= e) ; 
2 


where, it will be recalled; p, is the number of variables in the group G.. 
This formula, of course, agrees perfectly with (6.9) when the general 
residuals are considered as the observed correlations and p 
the total number of variables under consideration. ' 


It may be enlightening to apply formula (6.16) in the evaluation of one 


-factor 
is taken to be 
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group-factor coefficient of Table 6.1. Thus, the coefficient of F; for the 
fourth variable is given by 
E 1 Ba. . T 
a= o Os j kan je jn 4) 
2 


1 (fasta, Fastar ieta) 
d» aan Tu i 

The complete determination of a bi-factor pattern (6.12) has been shown 
to be possible by means of the formulas (6.14) and (6.16). When all the 
coefficients have been computed, the final residuals can be obtained. These 
are the residuals after all factors have been removed and may be written 


| as follows: 


(6.17) Fik = Tik — THe = Pik ada. 


If the variables j and k do not belong to the same group, then Fik = 7j. 
The general-factor residuals for variables from different groups, being the 
final residuals, must not be significantly different from zero. At the same 
time, the general-factor residuals for variables within a group must be sig- 
nificantly positive in order for an additional factor to be postulated among 
them, Sampling error formulas for the residuals are given later. 

When it is found that certain residuals between variables of different 
groups are significant, it may be necessary to modify the bi-factor plan 
slightly. The same is true if certain general-factor residuals within a group 
are practically zero. Some of these modifications will be illustrated later, 


after the sampling errors are considered. 


The method of analysis of this section is & : i 
tive correlations and yields a very parsimonious solution. The variables are 


Probably described in the simplest manner and with complete rigor and 
Statistical soundness. Furthermore, with the aid of the sampling error for- 
mulas which are developed in the next three sections, the eS E 
tell to what degree to factor a set of variables, i.e., determine the num A o 
factors that are necessary to reproduce the observed ees = in 
errors of sampling. To aid the reader in applying this E o a 7 de 
tion, a complete outline is presented in Appendix C. There the detaile 


ibi venty-four psycho- 
Tuan i ted on the battery of twenty 
Cy red ie The caleulations of the factor co- 


Ogical tests which has been used before. € ‘ 
efficients, by means of formulas (6.14) and (6.16), have been put in a routine 
form, 


daptable to any matrix of posi- 
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6.5. Standard Error of a Factor Coefficient 


The coefficients in a factor pattern, that is, the correlations of the vari- 
ables with the factors, may vary from sample to sample. Here the NE 
"sample" must include the variables as well as the group of individual * 
Inasmuch as factor analysis is essentially a correlational analysis, the entire 
table of correlations is subject to sampling error due to the sampling of in- 
dividuals and variables from their respective populations. It is evident, 
then, that any formula for the sampling error of a factor coefficient must 
be a function of the number of variables and the number of observations 
of each variable. . 

In this section the standard error of a general-factor coefficient will be 
developed. It will be assumed that the coefficient is obtained by the method 
of triads. This assumption is made for the sake of logical rigor because, as 
will be seen, the standard error formula depends upon the triads. Practi- 
cally, however, the formula will also be applicable to coefficients obtained 
by the method of 6.2, since such a coefficient differs very little from one 
evaluated by the method of triads. In addition, the standard error formula 


will be applied to group-factor coefficients after some justification for this 
is made. 


As a first step, it is necessary to determine the standard error of an in- 
dividual triad. Proceeding in 


the classical manner by taking logarithms of 
both sides of (6.8), there arises 


log tj = log r,; + log rex — log Tp 


Then, differentiating this expression, it becomes 
(6.18) dde db de 


a b e! 


where, for simplicity, the following substitutions have been made: 


(6.19) d E fre Teks c-r, 
Squaring the expression (6.18) and taking the mean value over the popula- 
tion gives 


2 2 2 
(6.20) f= “s+ z +54 2 (ten Tay — Ley s ) 
The purpose now is to r 


educe formula 
nal correlations a, b, an 


orn (6.20) to an expression in the origi- 
d c. This is accomplished by means of, first, the 
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well-known formula for the standard error of a correlation coefficient, 

namely, 
"EE CELE s 

(6.21) x liens ef 


where N is the number of observations in the sample; and, second, the Pear- 
son-Filon equations for the correlation between correlation coefficients. The 
form of these equations for the case where an index is repeated on the two 
correlations may be written* 


ab(1 — a? — b? — c? + 2abc) 
2(1 — a?)(1— b?) 


(6.22) Ta = C— 


In the derivation of these formulas it is assumed that the variables are dis- 
tributed normally in the population. Substituting expressions of the type 
(6.21) and (6.22) for the o’s and r's in (6.20), this equation becomes 


Notes e(= E+ ( z 2E (3 — eyl = on — b») 


c 


DU -U — c) , 2al — 09 e) 


ac bc 


tü-e-V-emb). 


By algebraic reductions this expression can be put in the form 


b. s. b ab? ab , ab? ab? ab? 
(423) Er a a a a T 


This formula gives the standard error of a particular triad. Formula (6.23) 
may also be written as follows: 


(6.24) No? = [ia " 2 di 5) +4420 - 9 1-2) + j] à 


which is a simple form for computation. 
In the above development only the variation in the sample of individuals 
was considered. When the sampling error of a communality or a factor 
* Karl Pearson and L. N. G. Filon, “On the Probable Errors of Frequency Constants 
and on the Influence of Random Selection on Variation and Correlation,” Phil. Trans. 
Roy. Soc., 1898, p. 263. 
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coefficient is desired, however, the variation in the sample of variables must 
be taken into account. Inasmuch as no method for dealing with the simul- 
taneous variation of individuals and variables is known, approximative 
methods are used. The general procedure, in making allowance for the vari- 
ation of variables, is to employ averaging methods. 

In obtaining the standard error of a communality, all variables will be 
considered by taking the mean of all possible triads. In the case of only one 
common factor, of course, each coefficient is merely the square root of the 
communality, that is, ao = Wh? (e = 1, 2,...,m). Hence the standard 
error of the coefficient can be readily deduced from the standard error of 
the communality by means of the formula* 


(6.25) 


1 
[X i e. 


According to the foregoing argument for developing an approximate for- 
mula for ca, it would be necessary to obtain the mean value of (6.24) for 
all possible triads and then apply (6.25). It appears to be somewhat simpler, 
however, to obtain the standard error of a particular evaluation of a coeffi- 
cient by means of (6.24) and then to take the mean value over all possible 
evaluations. Thus the value of the coefficient from the triad t is V t, and 


(6.26) v. 


1 2 
Typ Typ 0 
The formula for o?; in terms of the original correlations then becomes 


= ab ol" ,/b\? , ab ab ab\? 
(62) 4N&; = 2+ 244 2(2) (D) «m -stes(Ry. 


c c 


There now remains the problem to take the mean value of (6.27) over all 
possible triads. 

The terms in the right-hand member of (6.27) group themselves into 
five distinct types of nonlinear functions of two or three of the correlations 
a, b, c. The means of such functions can, in general, only be expressed in 
terms of the means, standard deviations, and correlations of the original 
variables (i.e., the correlations a, b, c) to a first approximation. Theimplicit 
assumption is that the deviations are small compared with the means of 


* For the general theorem relating the standard error of a function to the standard 
errors of its variables see Appen. B.T. 
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these variables. Designating the mean over v values by a bar, the means of 
the various types of functions in (6.27) may then be tabulated as follows:* 


ab ab GaU ab — FaFclac  ObOcTbe | Te 
mp pape SS Seo 
c é ab ac bc e 


(5 2 ( pue es desueTue 4 =) ; 


p E 
a” ac c^ 


(6.28) 2y Na ( 45, 555p QU, fence, imum 945) 
c 2 ü ab b ac bc ce 


2 2 
a a TaFslab , Cb GaUc ac , ObOcTbe , Oc 
-j2;(íl e cT om T oz Tu): 
bt āb b ac bc c 


ts 


Boece": of, =) 


ab\ _ ab GaU 8 aIcTac _ 
(2)-8(1+ ab dc bc e 


where the standard deviations and correlations are over the observed » 
values of the original correlations. In the derivation of these formulas it is 
assumed that the ratio of the deviation of a, b, or c from its respective mean 
to that mean is so small that third- and higher-degree powers can be neg- 


lected. 

The mean value of c2; over all possible triads will be written dä, to in- 
dicate the passage from an individual evaluation of a factor coefficient to 
the final value based on all » triads. Then, substituting the values from 
(6.28), and other expressions like these, for the mean values of the ratios 


in (6.27) gives 


; b oq 28° 4 al ae T E 


"rm € ü 20 20" 
IN Gi = 2 tota & € 8 z 
e (b 20 se) ei ( a 2E ae) 
«t ete EC 
ü b 3 b ab ab aR? 
EREE R E 
Me ac € ¢ ĉ é z 
(6.29) as ü "s b " ab y 5ab _ | 
~ d (i a č 6 e 
h "d ah ah 2272 
Tatelac ( à 5 8, 30 Sh , 100%") 
~~“ Me a € e Z e 
P^ "y ab ib 2252 
aefa 5, SH _ 90h E 
be \b ae C G é z 


A . B.8. 
* For the method of deriving these formulas see Appen 
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Formula (6.29) gives the complete expression for c2. to the degree of 
approximation indicated by (6.28). The complete formula, however, seems 
to be too long for practical purposes. It has been checked empirically that 
the first bracket gives a very good approximation to the complete expres- 
sion. The indication seems to be that the second bracket contributes a 
small positive quantity to the right-hand member. If the function of the 
correlations in the entire right-hand member is designated by f(r) and that 
in the first bracket by $(r), then the preceding observations may be stated 
as follows: 

f(r) > (r), 


or 


(6.30) fir) = or) +8, 


where 6? is a small positive quantity. 
Now the desired standard error is z,,,, so that it is necessary to express 
o%,, in terms of z2,. It is a well-known fact that 


Pa, 
where the standard deviations are for any variables. The last expression 
may also be put in the form 
(6.31) e=e+e, 


where € is a small positive quantity. Making the substitutions (6.30) and 
(6.31) in (6.29), there arises 


_ o(r) 
(6.32) Ba te= IN TL iy. 


The errors tend to compensate for one another, and the final formula is 
taken to be 


= 1 
(6.33) = py 000). 


Written out fully, and dropping the scoring over the g, this formula becomes 


(6.34) cà. = i x 48 b . 2à 2b , àb  5àb o. 
ĉ 


aeg p e 2 e == z "b a 
This represents the best approximation to the required formula for the 
standard error of a general-factor coefficient. 

To clarify the meaning of the terms in (6.34), suppose the number of 
variables in the set to be five and that the standard error of the first coeffi- 
cient of the general factor is required. The coefficient is denoted by aio and 
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R 5—1 P ; 
is evaluated from v = ( 2 ) = 6 different triads. Its solution, according 


to (6.9), is given by 
= fu. 
aly = ME = BD (MEE 5k = 23,45, F<). 


L \ Tjk 


The parameters defined in (6.19) are, for this case, 
t= ljr a = Tij, b= Tik, C = Tik 


(G, k = 2, 3, 4,5, j< k). 


The meaning of the terms in (6.34) will then be evident from Table 6.2. 


TABLE 6.2 
jk a-rnj b =rik c-rjk i= T 

DE siia. Ty: Tis Tas las 
24.. Ta? Tu Tu tu 
25.. Ti Tis Tas los 
34... Tis Tu Tu tu 
Fl cias Tis Tis Tss lss 
BD yet Tu Tis Tas lus 

Mean. ā b é ajo 
MEME E E 


Here all the correlations leading to the averages d, b, 6 are written down 
for clarity. In an actual problem, however, considerable time can be saved 
by omitting these three columns. The stub, giving the indices, should be 
retained to keep track of the variables, but this can be followed immediately 
by the column of triads. The appropriate correlations for computing à, b, ĉ, 
and the individual triads can be read off directly from the matrix of corre- 
lations, A numerical example, illustrating the computation of (6.34), will 
be given later. . 
Before leaving the present discussion, it may be well to point out another 
la (6.34). This formula was devel- 


application of the standard error formu : 
Oped on the assumption that the general-factor coefficients were computed 


by means of formula (6.9), i.e., the method of triads, As already remarked, 
formula (6.34) applies equally well when the general-factor coefficients are 
computed by the classical method indicated in 6.2. Now it would be very 
convenient if this standard error formula also applied to group-factor coeffi- 
cients. Such coefficients are obtained by the method of triads, as may be 
seen from formula (6.16). The essential distinction between (6.9) and (6.16) 
is that every correlation in the former is replaced by à general-factor residual 
in the latter. When the development of the standard error of a group-factor 
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coefficient was attempted, employing triads of general-factor residuals, a 
very unwieldy formula resulted. A very good approximation to the stand- 
ard error of a group-factor coefficient, however, is given by formula (6.34) 
in which the observed correlations are replaced by general-factor residuals. 


6.6. Standard Error of a Residual 


It will be recalled that a residual is defined by rj; = rj; — rjj, where 
rj; is an observed correlation and rj, is the corresponding correlation re- 
produced from a factor pattern. For the case of just one common factor 
rÍ, = ajoi;o and Fik = rjj — Ajodxo. The total differential for 7;, is 


drj = dr jx = ajodaxo = Axo ajo 1 
which, upon squaring and taking the mean value over the population, gives 
2 = ow 3 42 2 
(6.35) Cik T Orik F Ajo ako T Ao js T 2|aj011075;, Tag Taj, 
i95 


— A500 ra Doy P rjog, — 040774 Taj riis] + 


At the present time no formulas are known which express the corr 
tions in the bracket in terms of the original variables, ajo, axo, and rj» 
Hence the following expression 


a GLAD 2E 6 
(6.36) ory = Val, + aot, + alot, 


is the best approximation to the standard error of a residual, for one factor 


removed. The standard errors involved in formula (6.36) may be computed 
by means of (6.21) and (6.34). 


When two variables, z; and z;, involve two uncorrelated common fac- 
tors, Fo and F\, their residual correlation is given by 


(6.37) Fik = Tjk — 


Gjolyo — Aan. 

The result of differentiating and squaring the expression (6.37) and then 

taking the mean for the population is 

esci 2 g2 252 
Cj = Ct  GjoFa F dogs; + jig, + ahi; 

us 2(aj0 o 190 say aj F Ajoji ap, Taj Tags tpi 

(6.38) F Gj00 kiTa Taj, Taraj F Goo oj Tag ajoa 
is AKOAKO a; Taj T ajaj F ANANO ap, Taj Tapia; 
7 2509 rj. Fap eT rikako — AkOOr Oai rjkajo 


| — OnO0raCau Trias, — Q8 ra as Triaj) " 


THE BLFACTOR SOLUTION AND SAMPLING FORMULAS 129 


The best approximation for the standard error of the residual (6.37) is 
given by 


3r 2 T n P id. i og 
(6.39) epa = FF, dius + aod + aho, + Oa + 


an easily be extended to the general case. If two 


The foregoing results c 
m group factors in common, they may be de- 


variables have a general and 
scribed as follows: 


ayo + aF + aP +... + inl ms 
aroFo + anFı + awFet... + Bul a . 


z 
“i 


S 
ll 


Zk 


The final residual correlation is 


m 
(6.40) fik = Tik 7 X aran . 
s=0 
Then, as before, it can be shown that a good approximation to the standard 
error of this residual is given by 
m 
j 2 2 NS (at o2 2 gl 
(6.41) oj = Fix +> (a3,03,, + RaO) - 
s=0 
Ostensibly, formula (6.41) gives the standard error of any residual after 
t H H n z 
any number of factors have been removed, without any restrictions as to 
the type of factor pattern or method of computation of the coefficients. 
Such restrictions are implied, however, in the practical application of this 
formula. The formula involves the standard errors of the factor coefficients, 
which are not known for any type pattern but the two-factor and bi-factor. 
After further approximations, in the next section, à formula for the standard 
error of a residual will be obtained which does not —— the — 
errors of the coefficients explicitly. In that form i formu : ud A ap 
plied as an approximation to other types of factor so utions, for which the 


a ; , «nown. 
sampling errors of the coefficients are not kn 


6.7. Further Approximations to the Sampling Error Formulas 
i f a factor coefficient or a 
Other approximations to the sampling eprore Ob m. 
residual a be obtained by means of additional arbitrary, but reasonable, 
assumpti F cample, if a set of correlations is more or less homogene- 
adve csi ar any one of the correlations. Thus 


for 
ous, one may assume à constant value : 
A ; nay be set 
the correlations used in the evaluation of any factor coefficient may be 
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equal to their mean value, say p, yielding a2; = p. Under this hypothesis, 
formula (6.34) for the standard error of a cocfficient reduces to 


1. /3 
(ee el (ee > 2 
(6.42) a= 3N G 2 — 5p + 4p ) S 
which applies to any coefficient. The index a should not be confused with 
that defined in (6.19). Since this formula applies to all aso (e = 1,2,..., 
n), the subscripts have been dropped. 
Then, employing the value (6.42) for the standard error of a coefficient 


in (6.36) but letting the other elements retain their individual identities, 
that formula becomes 


(1 — 725? , a3 + a, /3 * 
(6.43) 3, = a E — 2 — 5p + ie) , 


For a rough estimate of the standard error of any residual, with one factor 
removed, an even simpler formula can be used. In place of the individual 
correlation r;, the average correlation p can be used, and, setting a?) = 
ało = p, formula (6.43) reduces to 


(6.44) eiim = 9X5 + Ba + 29) 


where the indices have been dropped from the residual correlation because 
this formula applies to any residual. 

Approximate formulas for the standard error of a residual, with two or 
more factors removed, can be obtained similarly. Substituting (6.42) for 
925, 02,,, and similar expressions for 955, 02, into (6.39), and setting rj, = P, 
aĵo = aĵo = p, and aj = ah = p, 


(1 — g*(5 + 8p +29 
(649 go PEE s 2 9n). 


where 


p — average correlation used in computing Fo coefficients, 
pı = average residual correlation used in computing F^ coefficients. 


In general, the standard error of a residual with (m + 1) factors removed 
is given by 


— (L= 9*6 + 8p + 2p?) 
(6.46) o = oN 


ié 
HERG- fot +200), 


THE BI-FACTOR SOLUTION AND SAMPLING FORMULAS 131 


where p, is the average residual correlation used in computing the F, coeffi- 
cients. 

Any one of the formulas (6.42) to (6.46) must be used with great caution 
and with a full realization of the underlying assumptions. Most important 
of all, one must check whether the assumption of setting all observed corre- 
lations equal to their mean is a tenable one. This assumption, of course, is 
over and above those assumptions made to get more or less practical formu- 
las like (6.34) and (6.41). Fortunately, the general direction of the dis- 
crepancies in using any of the approximate sampling error formulas is fairly 
well known. It has been verified empirically that at each stage of approxi- 
mation the values obtained for the sampling errors generally become small- 
er. As has already been pointed out, the standard error of a coefficient as 
given by (6.34) is usually smaller than that given by (6.29). Again, if (6.34) 
is approximated by (6.42) the value generally becomes a little smaller. Asa 
consequence of this, the standard error of a residual according to formula 
(6.46) is generally somewhat smaller than that obtained by formula (6.41), 
unless the original correlation is much larger than the average employed 
in the former formula. By knowing the general direction of the discrepancies 
in the approximations, the investigator can make due allowance in setting a 
level of significance.* 

With the foregoing limitations clearly in mind, it may still be very desir- 
able to use formula (6.42) and the various formulas for the standard error 
of a residual. For this purpose some of these formulas have been put in 
tabular form and are presented in Appendix H. The standard error of a 
coefficient for an average correlation from p = .10 to p = .75 and for 
samples from N = 20 to N = 500 is presented in Table H.1. For the same 
range of values of p and N Table H. 2 gives the standard error of a residual 
for one factor removed. Only one supplementary table is necessary, and, 
with very little computation, any value of (6.46) can beobtained. Table H.3 
gives the values of 

(Ẹ — p — $m + 2o). 


Thus, in general, the standard error of a residual with (m + 1) factors re- 
moved is obtained as follows: square the entry in Table H.2, for the par- 
ticular p and N, and add to this 1/N times the sum of the m entries of 
Table H.3, corresponding to the m values of p,; then the square root of this 
sum is the required standard error of the residual. 


* For example, if a particular residual is just twice its standard error (as given by one 
of the approximate formulas), it can safely be said that this residual is probably insignifi- 
cantly different from zero. The argument is that the standard error is probably a little 
larger, and the ratio a little less, than two. For such investigation, then, the level of sig- 
nificance should be taken to be at least three times the standard error. 
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The statistical tables of Appendix H, or the more exact formulas of 6.5 
and 6.6, may be used in determining the standard error of a factor coefficient 
or a residual correlation when such statistics refer to a bi-factor solution. 
For other types of factor solutions, which were presented formally in Chap- 
ter V and which will be described further in later chapters, the sampling 
error formulas do not apply. Since no sampling error formulas are known 
for other types of solutions, the formulas which have been developed for 
the bi-factor solution may be used, in such cases, as first approximations. 


6.8. Interpretation of Standard Errors 


The statement that a statistical constant (e.g., a factor coefficient or 
residual) has a value 7 with a standard error oz means that, if the constant 
be determined many times from random samples of relevant material, the 
observed value z obtained in a sample will differ in absolute value from 3 by 

~ less than c; in approximately 68 per cent of the number of samples. This 
conclusion is based on the assumption that the sampled values of z are dis- 
tributed in accord with the normal law; the value c; ceases to have the same 
significance if the distribution is not normal. 

The equation of the normal distribution of x may be put in the form 


(6.47) — ven eO 
TO 


where the origin is at the mean and the total area under the curve is unity. 
The values of z = z/s, or the number of standard deviations from the 


mean, are called deviates, and $a will be used to denote the area from the 
mean to such a deviate. Then 


(6.48) — pr SN assi, ee 
Tax, Jo uem h e "dv. 


The area $a gives the frequency of occurrence, or the probability of occur- 
rence, in the range from the mean to z/c. Table H.4 gives this total fre- 
Edi or probability integral, for values of a/c from 0 to 4 in intervals 
of .02. 

Frequently the question is to determine what fraction of the total popu- 
lation has a larger deviation than +2/c; or, in other words, what is the 
probability that a statistical constant so distributed, and chosen at random, 
will exceed a given deviation in absolute value. This probability is given by 
the areas in the tail pieces beyond +2/s, which may be written 


(6.49) P=1-a. 
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If the probability of exceeding +<x/o is required, it is, of course, given 
by 3P. 

The problem of greatest interest is to determine the significance of sta- 
tistics obtained from a given sample. A statistic is an appropriate function 
of the values of a variable given by a sample and used to estimate the corre- 
sponding population parameter, which is a hypothetical quantity helping 
to specify the mathematical form of distribution in the universe. Now an 
observed variance s? may be used as an estimate of the parameter c? of the 
distribution of x, or, better still, the mean value of s? from repeated samples 
may be employed. The variance c? refers to deviations from Z, whereas 
any s? refers to deviations from an z obtained in a particular sample. The 
value z may be considered as an arbitrary origin for a given sample, and 
the sum of the squares of deviates from this arbitrary point will generally 
be greater than the sum of the squares of deviates from the mean of the 
sample (by the least-square property of the arithmetic mean). Hence it is 
to be anticipated that the mean of the values of s* will be less than o?. The 
extent of this inequality is given by* 


N-1, 
E(s) = >y h, 


where E has been used to denote the expected value from repeated samples. 
It is advisable, especially when dealing with small samples, to use the un- 
biased estimate of the population parameter, which is defined by 

N 


Peg] 


N 


If N is large, the coefficient in the right-hand member approaches unity, 
and it is not invalid, to any appreciable extent, to use s in place of the un- 
biased estimate. 

When the “true” value of a parameter Z is known, and when the distribu- 
tion of this statistic in successive samples follows the normal law, the prob- 
ability of obtaining a deviation from the “true” value less than any given 
multiple k of the standard error of the parameter in question is given by 
the area under the normal curve inside the range + ko,. For example, if 
the observed value of a general-factor residual (say, x = .255) and its 
standard error (c. = .117) were the "true" values, then, referring to Table 
H.4 for the area, }a = .3413 corresponding to one deviate, it may be con- 
cluded that the probability is « = .6826 that in further samples from rele- 


* See Dunham Jackson, “Mathematical Principles in the Theory of Small Samples,” 


American Mathematical Monthly, 1935, pp. 344-64. 


“a 
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vant material the value of the residual obtained will be within the range 
.255 + .117. Similarly, it may be argued that the probability is a = .9544, 
or that in about 95 per cent of the samples the value obtained will be within 
the range .255 + 2(.117). These results may be stated in another manner. 
Under the same assumptions the residual obtained will differ from .255 by 
more than .117 in absolute value (i.e., it will lie outside the range of .138 
to .372) in 32 per cent of the cases; and it will differ from .255 by more than 
.234 = 2(.117) units in only 5 per cent of the cases. These are the percent- 
ages (to the nearest integer) of the total area under the normal curve outside 
of the respective ranges +o and +2c. Obviously, the greater the range 
adopted, the less likely it is that the sampled value will fall outside of it. 

When only the standard error, but not the parameter, is accurately 
known, the table of the probability integral may still be used to answer the 
question whether the observed parameter differs significantly from any 
given hypothetical value. This is usually the more important question. 
Thus, in the present illustration, it is not so important to know how close 
the value .255 is to the “true” value of the residual, but whether the ob- 
served value is significantly different from zero. Assuming that the stand- 
ard error .117 is the true value of that parameter, it is found that the devia- 
tion 0-.255 is 2.18 times the standard error. The area under the normal 
curve to the left and to the right of the range +2.18 is 


P = 1 — a = 1 — 2(.4854) = .0292, 


where ła = .4854 is obtained from Table H.4 for the ratio .255/.117 = 
2.18. This means that if the residual be determined many times from ran- 
dom samples of relevant material, for which the true value of the residual 
is zero, the value .255 would be exceeded in absolute value in 2.92 per cent 
of the cases, Even more important, however, is to test the hypothesis that 
the value obtained for the residual is significantly positive. In this case only 
the area to the right of 2.18¢ should be computed. The corresponding 
probability is 
iP = }(.0292) = .0146, 


which means that in sampling the residual, from an assumed true value of 
zero, the value .255 would be exceeded positively in only 1.46 per cent of 
the time. Consequently, one may conclude that the true value of the residu- 
al is different from zero or, for the second hypothesis, that the residual .255 
is significantly positive.* By the same procedure the significance of the de- 
parture of the residual from any other hypothetical value could be tested. 


* of course, if there is sufficient evidence, on other than pure statistical grounds, for 
the validity of the hypothesis that the true value is zero, then it would not be rejected. 
One might argue, however, that the particular sample was a very unusual one. 

‘ 


= 
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In drawing conclusions, the investigator may be more or less exacting in 
the smallness of the probability that he would require. "It is usual and 
convenient for experimenters to take 5 per cent as a standard level of sig- 
nificance, in the sense that they are prepared to ignore all results which 
fail to reach this standard, and, by this means, to eliminate from further 
discussion the greater part of the fluctuations which chance causes have 
introduced into their experimental results."* With this standard for the 
level of significance, an investigator would attribute to “pure chance" re- 
sults having probabilities up to 1 in 20. In the preceding example the odds 
against the conclusions on à pure random sampling chance are .9708 to 
.0292 or 33 to 1 in the first case, and .9854 to .0146 or 67 to 1 in the second 
case. It is thus seen that the conclusions were drawn in accordance with 
common practice. 

Although the foregoing procedure cannot, strictly speaking, be applied 
to parameters of factor analysis because the standard errors are not accu- 
rately known, it may be used when N islarge. In factor analysis the samples 
are usually large, and for this reason it is suggested that the ratio of the de- 
viation (of an observed statistic from an assumed value) to the standard 
error be referred to a normal probability scale. It should be emphasized 
that, when the “true” value of the parameter is taken to be zero and the 
preceding test is applied, there are really two hypotheses being tested: first, 
that zero is the true value and, second, that the parent distribution from 
which the sample is drawn is a normal one. An extremely high value of the 
ratio might then suggest either that zero is not the “true” value of the 


parameter or that the distribution is not normal.t The conclusion would 


be that the observed value of the parameter is significantly different from 


zero or that the assumption of a normal parent population was unjustified. 

To guard against drawing erroneous conclusions, the investigator must 
supplement his purely statistical tests of significance with all the theoretical 
and factual knowledge at his disposal. Thus, the general-factor residuals 
among a set of variables, which were shown to belong together by the 
B-coefficients, may all turn out to be positive but not very significantly 
different from zero according to the purely statistical tests of significance. 
In this case, although it might be argued that the residuals are insignificant- 
ly different from zero (with perhaps probabilities of only up to 1 in 20 
against pure random sampling chance), such a hypothesis would be re- 
jected. The very consistency in positive values would lead the investigator 
to doubt that every one of these residuals was really a sampling fluctuation 


* R. A. Fisher, The Design of Experiments, 1935, pp. 15-16. 


1 The assumption of normality is made, for convenience, in practically all sampling 


problems. " 
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from zero, always in the same direction! A group factor would be postulated 
in such a case, and the final residuals for this set of variables would be ex- 
pected to vary positively and negatively around zero. 

The standard errors derived in this chapter must be used with great 
caution in drawing probable inferences, for they are, at best, only approxi- 
mations to the true values. Furthermore, the tests of significance imply a 
normal parent population from which the sample is drawn. The form of 
distribution of a triad, a factor coefficient, or a residual in successive samples 
is not known; for, even if the correlations were normally distributed, the 
same could not be said, a priori, of these nonlinear functions of the correla- 
tions. It is evident, then, that the standard errors of factor analysis do not 
have the same heuristic properties that they are presumed to have in the 
natural sciences. The standard errors enable the investigator to answer the 
question, “If the observations constituting the sample are drawn at random 
from a normally distributed universe, and if the true value of the parameter 
is Z, what are the odds against the observed value z having arisen as a result 
of the fluctuations of sampling?" In this role the standard errors become 


one of the measures of goodness of fit and, as such, are extremely useful 
and often indispensable. 


6.9. Verification and Modification of the Bi-factor Patlern 


The sampling error formulas may be employed for the same purposes as 
such formulas are used in other branches of statistics. Thus, the significance 
of any factor coefficient or residual may be tested. Again, if two factor pat- 
terns, based upon two samples of individuals, are obtained, the difference 
between any two factor weights may be tested for significance. Some other 
applications of the sampling error formulas suggest themselves for factor 
analysis. Formula (6.24), for the standard error of a triad, may be used to 
justify the conditions for one common factor. This formula may also be 
applied at the second stage of analysis, i.e., when the general factor has been 
removed and a certain set of general-factor residuals are considered as 
measuring one group factor, 

The refined check for one common factor among a set of correlations may 
be obviated, however, by employing the B-coefficients, dropping all triads 
which exceed unity, and observing the table of intercorrelations for in- 


significant correlations for an entire row and column. If several variables 


apparently form a group, but the set of general-factor residuals for that 
group includes a row and column of insignificant values, the variable with 
these residuals should be dropped from the group and should not be used to 
measure that group factor. Such a variable then measures only the general 
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factor, and its residuals with all other variables are then final residuals. On 
the other hand, if some of the final residuals turn out to be significantly 
different from zero, further factorization is indicated. Variables producing 
such residuals have a factor in common, and the pattern plan must be modi- 
fied to include these changes. If a new factor is indicated for a subset of 
variables, then the preceding group-factor coefficients for these variables 
must be modified, for the hypothesis that these variables had involved only 
that one group factor is then invalidated. In computing the common-factor 
variance, beyond the general factor, the variables must be assumed to have 
a correlation matrix of rank two rather than rank one. The important thing 
is to formulate the new pattern plan; once this is done, the actual computa- 
tion of the factor coefficients is a simple matter. First, triads are selected so 
that no two variables measure more than one common factor. Then formu- 
la (6.9), with the appropriate triads, may be used to calculate the coeffi- 
cients. If the overlapping between variables is so great that appropriate 
triads cannot be selected, then a formula of type (4.22) may be used and 
the coefficients determined as in 4.7. 

The procedures for modification suggested in the last paragraph may 
seem to be somewhat subjective. Nevertheless, it can be done very quickly 


the more exact sampling error formulas. The significance 


and checked by 
al tables in 


of a coefficient or residual may be determined from the statistic 
Appendix H; or, in particular instances, formulas (6.34) and (6.41) can be 
applied. Not only particular residuals but the arrangement of the entire 
set for a variable, or several variables, may give a clue for modification of 
For example, if there is one large residual while all 
given variable, a doublet (factor through only 
ated between the two variables producing 


the factor pattern. 
others are insignificant for a 
two variables) may be postul 


that residual. 
It should be noted from Table 4.2 that, when only two variables are as- 


sumed to measure a factor, their communalities can have an infinite number 
of solutions, so that it requires at least three variables to determine the 
factor weights uniquely. The two variables may therefore be considered as 
indicating a potential factor, which might h 
of that factor were present. In determining the doublet weights for the 
given variables, one standard deviation of the residual may be arbitrarily 
selected as chance error and the remainder divided equally between the 
two variables. Thus, if the one large general-factor residual is between 
2; and z;, and the doublet is denoted by Di and its coefficients by dj, da; 


then 


ave appeared if more measures 


(6.50) da = du = Vig — %- 
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For the example of the last section, if the significant residual 7;, = .255 
conformed to the conditions for postulating a doublet, the coefficients of 
variables j and k for this doublet would be 


da = dy = V.255 — 117 = V.138 = 311. 


The final residual would be 7;; = .117, which, of course, is insignificantly 
different from zero. 

Special attention should be paid to the analysis after the general factor 
has been removed. Slight modifications in the grouping of variables and 
new factors between several variables will not change the general-factor 
coefficients very much. Each of the general-factor coefficients is based on a 
large number of correlations so that a few changes in the pattern plan have 
little effect on them. A few discordant triads in the calculation of a group- 
factor coefficient, however, may greatly affect its value. For this reason, no 
triad should be used which exceeds unity. Also, no triad should be used 
which involves insignificant general-factor residuals. One other method 
might be used in the verification of variables measuring a group factor. The 
method of B-coefficients may be applied to general-factor residuals for pur- 
poses of verifying the elimination of certain doubtful variables from the 
original group. The B-coefficients for combinations of two or three vari- 
ables, including the doubtful one, may be calculated and their magnitude 
inspected for the degree of belonging together. 

The preceding discussion is mainly concerned with modifications of the 
bi-factor pattern when the residuals seem to indicate that higher com- 
plexities are required for some variables. By verifying the significance of 
particular factor coefficients, the complexity of a variable may be reduced. 
Thus one or two general-factor coefficients might be of the order of their 
standard errors and in further sampling they might be zero. In the modified 
bi-factor pattern these variables would be dropped from measuring the gen- 
eral factor. Hence the final solution would not involve a general factor. 
The blankness in a bi-factor pattern can similarly be justified by means of 
the standard error of a factor coefficient, for, if a coefficient were computed 
wherever there is a blank, its value would be insignificantly different from 
zero. 

An illustration of one type of modification will now be given. Table 6.3 
contains the observed intercorrelations of five physical variables, taken 
from a total set of seventeen and based upon N = 305 fifteen-year-old 
girls.* The complete analysis of the seventeen variables is of no concern 


* These data are taken from Frances Mullen, “Factors in the Growth of Girls Seven 
to Seventeen Years of Age," 1939. 
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here, the particular portion having been selected to illustrate a point. A 
complete bi-factor solution will be exhibited in the next section. The pres- 
ent example of five variables is introduced to clarify some of the preceding 
ideas on the application of sampling error formulas and modification of a 
bi-factor solution. 
TABLE 6.3 
INTERCORRELATIONS OF FIVE PHYSICAL VARIABLES 


Variable 1 2 3 4 5 
1. Height. sse ees] een poser enee}eewie ene 
2. Span of arms...... 4848. EE rnnt fta coena trea cc 
3. Length of forearm..| .805 eB E, pte cmo Imre mc 
4. Length of lowerleg.| .859 .826 SOL [aeuo scndiitmm nas 
5. Sitting height. .... .740 .497 .494 ABOU hreins 


In the total set of seventeen variables the five variables of Table 6.3 
were found to belong together by the method of B-coefficients. The bi-factor 
pattern plan was then assumed to contain a general physical (or growth) 
factor, say Fo, and a group factor, say Fi, through these five variables. The 
general-factor coefficients were calculated by the method outlined in Ap- 
pendix C, and for the given five variables these coefficients are aio = .691, 
az = .591, ago = .581, dso = .598, and aso = .674. The general-factor re- 
sidual correlations among the five variables are presented in Table 6.4. 


TABLE 6.4 


GENERAL FACTOR RESIDUALS AMONG 
FIVE VARIABLES 


Variable 1 2 


4454. Beene ee 
.102 | .048 |....-- 


The immediate problem is to determine whether the original hypothesis 


that the five tests measure a single group factor is warranted from the na- 
ture of the residual correlations in Table 6.4. A general inspection of Table 
6.4 seems to indicate that variable 5 does not exactly fall in line with the 
other four. It seems quite likely that variable 5 has additional linkage with 
variable 1, beyond the general factor, but that no further linkage with the 
other three variables is necessary. In other words, the general-factor residu- 
als of 5 with 2, 3, and 4 might very well be final residuals. This conclusion 
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will be checked by various methods in order to exhibit the procedures for 
modification of the pattern plan. 

First, the method of B-coefficients will be applied to the general-factor 
residuals to verify the elimination of the doubtful variable 5 from the origi- 
nal group. The B-coefficients for combinations of 5 with the other variables 
are presented in Table 6.5, where the notation employed is that of Chap- 
ter II. The value B(1, 5) = 107 indicates that the variables 1 and 5 belong 
together to just about the same extent to which they belong with the other 
three variables. All the remaining B-cocfficients of 5 with the other vari- 
ables are under 100 and clearly indicate that 5 does not belong in the group. 


TABLE 6.5 
B-COEFFICIENTS BASED ON GENERAL-FACTOR RESIDUALS 


| ES 200(n—p)8 

u | P ^ | EMT T (p-1)T Blu) = (p-DT 
BV RE | £2 | J274 600 1.537 1.537 107 
(2,5). 2 .099 600 1.873 1.873 32 
(CELO MANN 2 .102 600 1.817 1.817 34 
(C81: MAMMA, | 2 048 600 1.848 1.848 16 
LORD. sas sccm aes pd 19 .811 400 2.011 | -4.022 81 
(IAS Dosta 2 .T80 400 2.023 4.046 77 
UL B). i aana l B -768 400 1.970 | 3.940 78 
(0,8 DE E iy ven: 3 -739 400 2.091 4.182 71 
(2H B exscr d .620 400 2.252 4.504 55 
Bj E uersa d .604 400 2.234 4.408 54 
(953458). E 1.714 200 1.562 | 4.686 73 
(ip BBs NM | 4 | 2.753 200 .523 1.569 164 


The last entry, B(1, 2, 3, 4) — 164, is included in Table 6.5 for contrast. 
There is no doubt that variables 1, 2, 3, and 4 belong together, while vari- 
able 5 probably does not belong with this set. 

The modification of the pattern plan implied by the preceding analysis 
is to drop variable 5 from measuring the group factor F,. The entries in the 
last row of Table 6.4 would then be final residuals. Now the final residuals 
must be zero except for errors of sampling, so that these residuals should be 
tested for significance. For purposes of comparison, the standard errors of 
these residuals have been computed by formula (6.36) and also from Table 
H.2 and are listed in Table 6.6 together with the ratios of the residuals to 
their standard errors and the probabilities that, in random sampling, the 
observed deviations of the residuals from zero would be exceeded in absolute 
value. The probabilities that the observed residuals would be exceeded in 
the positive direction only are just one-half the respective probabilities 
given in the table. 

In applying formula (6.36), the standard errors of the general-factor co- 
efficients are required. For the present example the approximate value s, = 
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.066 from Table H.1, corresponding to an average correlation p — .355* 
and N — 305, was employed. Another numerical comparison of standard 
errors, in which the standard error of a coefficient is computed by means of 
(6.34), will be given in the following section. The two sets of standard errors 
in Table 6.6 are very little different. 

The probability P for the residual, = .274 is zero to four decimal places 
and clearly indicates that in sampling, assuming a true value of zero, the 
observed value would be exceeded in less than 1 per cent of the cases. It 


TABLE 6.6 
STANDARD ERRORS OF RESIDUALS 


Bv FonwuLA (6.36) From TanLE H.2 


j ts ei fep |Pe1-a| oF | Tes 


.274 | .069 | 3.97 | .0000 | .074 3.70 | j 
‘099 | :073 | 1:36 | :1738 | -074| 1.34 | 1802 
0.65 | f 


.102 .078 1.40 | .1616 .074 
.048 .075 | 0.64 | .5222 .074 


may then be safely concluded that the true value of this residual is different 
from zero. The probabilities for the remaining three residuals are each 
greater than .05, the standard level of significance recommended in the last 
section, and so the deviations of these observed values from zero may be 
attributed to chance errors. In other words, the odds against the residuals 
being different from zero by “pure chance" are all less than 20 to 1; there- 
fore, the conclusion that the true values are zero is acceptable. 

Now the nature of the factor pattern plan of the five variables which was 
conjectured earlier from an inspection of Table 6.4 has been verified by the 
more objective statistical procedures. In accordance with these findings, 
the new pattern plan may be formulated as in Table 6.7. 


TABLE 6.7 


NEW PATTERN PLAN FOR 
FIVE VARIABLES 


Variable Fo Fi Di 
aio au 
20 an 
Q30 Q3 
Go aa 


* Computed from the intercorrelations of all 17 variables; see ibid., Table 9. 
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The coefficients a (e = 1, 2, 3, 4) may now be computed by means of 
formula (6.16), which reduces to 


ag D (b ik-nisa ien jke) 


Tik ! 


for the present case. They are as follows: 


au = V3(.32891 + .41300 + .39688) = v/37960 = .616 , 
an = V4(.58328 + .46452 + .56052) = V.53611 = .732, 
aa = V4(49624 + .41125 + .51639) = V.47463 = .689, 
a4 = V3(.48164 + .50120 + 39915) = V.46066 = .679. 


It is evident that the triads involved in the calculation of any one of these 
coefficients are sufficiently constant so that the conditions (4.13) for one 
common factor are satisfied by the intercorrelations of the first four vari- 
ables in Table 6.4. The hypothesis of the single factor F, through these 
four variables is then quite adequate. 

There remains the calculation of the doublet coefficients in the plan of 
Table 6.7. If, as indicated earlier in this section, one standard deviation is 
allowed for chance error and the remainder of the residual correlation .274 
is divided equally between variables 1 and 5, then the communality of 
variable 1 exceeds unity, implying an imaginary unique factor. To circum- 
vent this difficulty, allow only one-tenth of the total variance of variable 1 
to the doublet D,, making its coefficient dj, = .316. This arbitrary pro- 
cedure is permissible as one of the infinite variety of ways in which the 
coefficients of a doublet can be assigned. The other coefficient then becomes 


= :274 — .069 — 
da = — sia c 649 . 


The final factor pattern, 


for the given five variables, may now be writ- 
ten as follows: 


A = .691F, + .616F, + .316D, 
23 = .591F + .732F, 
23 = .581F, + .689F, 
2, = .598F) + .679F, 
Zs = .674F + .649D, 
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where, as usual, the unique factors have been omitted and the variables 
have been designated by z's without double primes.* All the final residuals, 
with the factors Fo, F;, and the doublet D; removed, are insignificant so 
that no further factorization is required. The modification of the original 
bi-factor plan exhibited in the present example is, perhaps, one of most fre- 
quent occurrence. The analysis is simple and direct and was given in such 
detail here merely to bring out and clarify various procedures. In practice, 
the investigator would come to a conclusion as to the nature of the new pat- 
tern plan much more quickly: he might, at times, apply the B-coefficient 
technique; at other times, the tests of significance; while still more often, 
he might be able to formulate the new pattern plan simply by inspection of 
the general-factor residuals. 


6.10. Bi-factor Solution for Twenty-four Psychological Variables 


To round out the ideas of this chapter, a complete bi-factor solution for 
the data of 2.8 will be presented. The details of the straightforward analysis 
are given in outline form in Appendix C, while some of the refined tech- 
niques of modification and tests of significance will be discussed in this sec- 
tion. For purposes of reference, and to avoid repetition, the complete final 
factor pattern is given in Table 6.8. The manner in which the factor coeffi- 
cients were obtained will be explained in the sequel. 

In the outline of calculations of Appendix C the order of analysis is de- 
scribed in detail, beginning with the grouping of tests from the original 
matrix of correlations. This leads to step 17 in which the general-factor 
coefficients are calculated, and these are the values in column B, of Table 
6.8. The next stage of analysis involves the elimination of the general fac- 
tor, yielding residuals from which the group-factor coefficients are calcu- 
lated. The coefficients of Bı, B», and Bs are computed in steps 21 to 23. 
It is evident from Table C.6 that the general-factor residuals among the 
tests of Gs are insignificantly different from zero and so may be considered 
as final residuals, obviating the original hypothesis of a group factor for these 


tests.t This leaves the group G; to be analyzed. 


* See n. *, p. 95. 

+ With this modification of the original pattern plan, it is evident that there are addi- 
tional terms which can be included in formula (6.14) for the calculation of any one of the 
general-factor coefficients. These additional terms arise because the tests of G; measure 
only the general factor and no group factor. Thus any two of the Tests 20 to 24, together 
with any other test of the battery, involve only one common factor. Since each general- 
factor coefficient is based on a large number of correlations, however, the addition of a 
few more terms into the average of more than & hundred terms will not change its value 
much. For this reason it is not necessary to recalculate the general-factor coefficients. 
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From an inspection of the general-factor residuals among the tests of 
G, it would appear that Tests 14 to 19 do not measure a single common 
factor. Instead, it would seem that 17, 18, and 19 belong together; 14, 15, 
16, and 17 involve another factor; and the residuals for 14, 15, and 16 with 
18 and 19 raight be final residuals. These changes will now be justified. 


TABLE 6.8 
BI-FACTOR PATTERN FOR TWENTY-FOUR PsYCHOLOGICAL TESTS* 


General | Spatial Percep- | p Associ- 
Testh Deduc- | Rela- | Verbal | tual TO08- | ative | Doublet | Unique 
tion | tions Speed | "tion | nremory! 
j Bo U; 
ls .589 .647 
2.. .357 .889 
3.. .401 .781 
LM .463 .828 
5.. .582 .576 
B. .575 .597 
Miss .534 .463 
8.. .624 .686 
9. .560 .540 
10.. .388 .599 
El. .521 .707 
12.. .404 .652 
18.. .576 .690 
14.. .388 .743 
15.. .351 .806 
16.. .496 .793 
TE. .422 .670 
18.. .515 .718 
19.. .442 !858 
20.. .644 1765 
2]. .645 1164 
22. .644 "165 
23. -734 .679 
24 .712 .596 
Contribution of 
Tactor......| 6.874 | 0.645 | 1.678 | 1.185 | 0.779 | 0:538 | 0.375 |.....<; 


From step 24 in Appendix C it is evident that the general-factor residuals 
among the Tests 14 to 19 do not form a matrix of rank one. It will next be 
shown that the residuals for Tests 14, 15, 16 with 18 and 19 are insignifi- 


of these six residuals is fisig = .158; 
o by chance errors of sampling, then 
d as such. To make this statistical 
al is required. This can be obtained 
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from Table H.2 of Appendix H for N — 145 and p — .30* by linear inter- 
polation giving c; = .113. The ratio of the residual to its standard error is 


T ot = 126, 
O; " 


for which the probability P — .1770, obtained by linear interpolation from 
Table H.4. This result means that in sampling, assuming a true value of 
zero, the value .153 would be exceeded in absolute value in 17.70 per cent 
of the cases, or that the observed value is not significantly different from 
zero. Stated another way, the odds against the true value of the residual 
being different from zero by "pure chance" is only .8230 to .1770 or 4.65 
to 1, so that it is quite likely that the deviation from zero is due to random 
sampling. Since the largest of the six residuals is insignificantly different 
from zero, it may safely be assumed that all six are insignificant. Hence no 
linkage beyond the general factor is required between any one of the Tests 
14, 15, 16 with 18 and 19. 

The modification of the original hypothesis for the tests of G now clearly 
points to a factor for Tests 14, 15, 16, and 17, and another factor for Tests 
17, 18, and 19. This conclusion is also corroborated by the B-coefficients 
for these two subsets of tests, namely, 


B(14, 15, 16, 17) = 202 and  B(17, 18, 19) = 177, 
where the entire set of general-factor residuals for the six tests was em- 
ployed. The new pattern plan may then be written as in Table 6.9. 


TABLE 6.9 
NEW PATTERN PLAN FOR 
SIX VARIABLES 


Test Bo Bı Bs 
jc T au, o ua esee 
17 MEN 15,0 Os exe voe 
TO hs nasse dx 16,0 (wa — [xs een 
ip m Q0 Q4 ys 
I8. eee üe een eene 18,5 
Tia ssia G1,0 [arei gise 19,5 


* The average correlation is obtained from the entire set of raw correlations of Table 
2.2by summing all the column sums and dividing by twice the number of different corre- 
lations. In symbols, this is 


: 166.298 
Zimi j,k =1,2,..-, 24, jAk) =g = 301. 


ER, 
^7 205 
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According to the revised plan, the general-factor residuals for Tests 14 
to 19 are assumed to form a matrix of rank two. Now the direct method of 
analysis, as exhibited in 4.7, may be employed for this case; or else appro- 
priate tests may be selected, from the total set of six, which taken alone 
have a correlation matrix of rank one, and then the method of triads may 
be applied. The solution by the method of triads is given in Appendix C 
and the coefficients recorded in columns B, and B; of Table 6.8. For com- 
parison, the solution has also been obtained by applying formula (4.22) and 
the procedure of 4.7. The coefficients obtained by the latter method are 
listed in Table 6.10 under Bj and B/ to distinguish them from the values 
obtained by the former method. A comparison of the two sets of coefficients 


TABLE 6.10 
COMPARISON OF Two SETS OF B, AND B; COEFFICIENTS 


is also given in this table. The differences are obviously insignificant, and, 
because the values are obtained more easily by the method of triads, these 
are used in the final pattern. 

The original correlations are now accounted for by means of the general 
factor and five group factors except for one* large residual, 710,24 = .255. 
To test this residual for significance, its standard error is required and will 
be computed by means of formula (6.36) and compared with the value from 
Table H.2. A similar comparison was made in Table 6.6, but in using for- 
mula (6.36) the standard errors of the coefficients were obtained from 
Table H.1. Now formula (6.34) will be used to get the standard errors of 


the factor coefficients, thus illustrating the computation by means of this 
formula. 


* There is one other large general-factor residual, namely. 
would be no point in applying statistical tests of significance 
of the problem clearly indicate that this value m 
lation was only ra, 19 = —.075, and it is due to re 
tion of the general-factor coefficients that the r 
tively. 


73,10 = —.231. But there 
to this value, for the facts 
ust be insignificant. The original corre- 
placing this value by zero in the calcula- 
esidual for these tests is increased nega- 


hs md 


— — —Á—— Ver SS 
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First the standard error of a, Will be eoleulated. It will be recalled 
that there are n — 24 variables and that there are 


m=4, p=5, ps=4, | p»4—0, Pps=5 


variables in the respective groups G, to Gs. Now 10eG; so that the system 
((10, j, k) ; jeG,, keG,, s < t, s, t= 1, 2, 4, 5) may be used in evaluating 10,0. 
The number of triads which can be set down frcm such triples is easily 
found to be 

= pi(pa + pa + ps) + p(pa + ps) + pals) 5 

= 4(16) + 5(11) + 6(5) = 149. 


The component parts of triads formed from such sets, designated by a = 
Tioj, b = rios, C = rj; according to (6.19), must be averaged over the 
v values. It is necessary to write out neither the individual values of a, b, 
and c, as was done in Table 6.2, nor even the indices and triads for the pres- 
ent example because the coefficients have already been computed in Ap- 
pendix C and the necessary sums of correlations can be picked out of Tables 


C.1 and C.2. Thus, 


x 


i 1 : P " 
a= [p pe p:)E(rio,i5J683) + (pa + p3)E (19.553682) + psE(ro,53€G4)] 


ü i [16(.272) + 11(1.215) + 5(1.246)] 
_ 23,947 _ 
= ap = 1907, 


where the appropriate sums of correlations for Test 10 with the tests of 
groups Gi, G2, and G, are obtained from row 10 of Table C.1. Similarly, 


x 1 
Ü F [p1Z (710, x; keG2) + (pit p2)Z 19,5 keG4) + (pit pa p3)Z (710,45 keG's)] 


5 i5 [4(1.215) + 9(1.246) + 15(1.531)] 
39.039 
“Tag = 2020 


The correlations c = rj; are precisely those involved in the denominator of 
formula (6.14) for the evaluation of.a factor coefficient. Hence, employing 
the denominator for 10c@s from Table C.2, 


42.348 _ 
ng = 2842. 


t= 
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Now, substituting the values of à, 6, and c into formula (6.34), the standard 
error of the coefficient a1», becomes 


[LL ida (2 + 2.15821 + 5.73668 — .63945 — 1.69974 + 1.83415 
— .74073 4- .08779) 


and 
Fay, = 129. 


Before the standard error of the residual 710,23 can be calculated by for- 
mula (6.36), the standard error of aj, must also be obtained. Proceed- 
ing as before, there are 


v = pi(ps + pa + ps) + pipa + pi) + papa) 
4(15) + 5(10) + 4(6) = 134 


sets from which triads may be formed in evaluating the general-factor coeffi- 
cient for 24«G;. The average of the correlations a = T24,; is given by 


1 r 4 
a= T [ps + pa + pa)E(r24,5; jeG1) + (ps + p3)Z(ro,j; je@o) 

F Ps2(724,;; JeG3)] = .3393 , 
where the sums are obtained from row 24 of Table C.1. In like manner, the 
average of the correlations b = T24, İS given by 
s 1 
b= 3 [piZ (724,5; keGe) + (pi + D2)Z(r4,4; keGa) 

+ (pi + po + Ps)Z(re4,43 keG4)] = .3561 . 


From Table C.2, the denominator for any test of Gs is 31.455, so that 


The square of the standard error of the coefficient az4, then becomes 


a es 
Ta, o Ps 


2 + 4.05973 + 4.47171 — 4.17994 — 4.60413 + 9.34599 — 2.57408 + 1.06010 
4(145) 


= .01652, 
and 


Tas = 129. 
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Now the preceding values for the squares of the standard errors of the 
factor coefficients may be substituted into formula (6.36), giving 
_ (0 — 85312? 


Wd = + .15036(.01652) + .50705(.01506) , 


[^4 


where the first term is the square of the standard error of r1o,24 = .531 as 
given by (6.21). The standard error of the required residual is then 
v5, = V.01368 = .117. 

Previously, on page 145, the value for the standard error of any general- 
factor residual, according to Table H.2, was shown to be c; = .113. There 
appears to be a remarkably close agreement between the value obtained by 
means of formula (6.36) and that from Table H.2, just as in the case of the 
example of the last section. 

The significance of the residual io,» = .255 can now be tested, knowing 
that its standard error is .117. As a matter of fact, these figures formed the 
basis of the illustrations in 6.8, where it was shown that the residual .255 
was significantly different from zero. Since all other residuals, with the gen- 
eral and the group factors removed, for Tests 10 and 24 are insignificant, a 
doublet may be postulated between these tests. Again, in 6.9, the doublet 
weights were shown to be 


dioi = doa = .371 PI 


This completes the analysis into common factors. 

It may be noted that there are no insignificant values in Table 6.8. A 
rough estimate of the standard error for any one of the coefficients may be 
taken from Table H.1 for N = 145 and p = .30, namely, v; = .109. The 
smallest coefficient is .278, which is 2.55 times this standard error, while 
any one of the other coefficients is more than three times its standard error. 
Hence every coefficient in the final factor pattern is definitely significant. 

The unique-factor coefficients can readily be obtained by applying the 


formula 
a; = V1— HM, 


where A? is the communality of test 2;. The com i : 
each test are given in Table 6.11, while the square root of the uniqueness 18 
entered in Table 6.8 as the coefficient of its unique factor. Table 6.11 also 
contains the unreliability and the specificity so that the apportionment of 
the unit variance of each test can be seen at a glance. In addition, the index 


munality and uniqueness of 
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of completeness of factorization is also given for each test. This shows the 
percentage of the reliability variance accounted for by the common factors. 

The index H; may be used as one standard for the adequacy of a factorial 
solution. The analysis of psychological tests into common factors should 
not be carried to the point where real specific factors disappear. In the pres- 


TABLE 6.11 
APPORTIONMENT OF TEST VARIANCES 
= ia- " Unrelia- Per Index of Fac- 
Test pe pow Uniqueness | "piis | Specificity torization 
ei DH ris aj-1-M | ejel-rj, | $jeal-d Hj e 10077 

Tu. .581 .756 .419 .244 m] 76.9 

2. .209 .568 .791 .432 .359 36.7 

3.. .390 .944 .610 .456 .154 (Aes 

4.. .315 .922 .685 .078 .607 34.1 

5.. .668 .808 .332 .192 .140 82.7 

6... .643 .651 .357 .349 .008 98.8 

Mss .786 .754 .214 .246 —.082 104.2 

en .530 .680 -470 .320 .150 71.9 

9.. .708 .870 .292 .130 .162 81.4 
10.. -503* .952 .497 .048 .449 52.8 
JUPE .500 .712 -500 - 288 .212 70.2 
32.. -575 .937 .425 .063 .362 61.4 
13.. .924 .889 .476 .111 .965 58.9 
14.. .448 .648 .552 .352 .200 69.1 
15.. .950 -507 .650 .493 .157 69.0 
18.. .371 .600 .629 .400 .229 61.8 
1T.. -551 725 .449 .275 174 | 76.0 
18.. .484 .610 -516 -390 .126 79.4 
19.. -273 .569 727 .431 .296 47.9 
20.. .415 .649 .585 .351 .234 63.9 
21.. .416 -784 -584 .216 .368 58.1 * 
22.. .415 .787 .585 .218 .372 52.7 
23.. .539 .931 .461 .069 | 1802 57.9 
24 .5075 .836 .493 .164 329 60.6 


* The communality with the doublet Di included is .641, 
b The communality with the doublet Di included is 645. 


ent example, there is one value of H i exceeding 100 per cent, and which is 
probably due to chance. The preceding checks, and the one to follow, indi- 
cate that the factorization has not been carried too far, so that this dis- 
crepancy may be attributed to chance errors in the reliability coefficient. 


Only when several values are greater than 100 per cent, for high reliability 
coefficients, should the factorization be changed.* 


* An example where even this crud 
carried too far is available in Thurs 
nalities for seven tests appreciably e: 


e test would indicate that the factorization has been 


tone, Primary Mental Abilities, where the commu- 
xceed the reliabilities. 
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As a final check on the adequacy of the factor pattern, the criterion of 
2.5 will be applied. This requires the standard deviation of the series of 
final residuals to be less than, or equal to, the standard error of a zero corre- 
lation for a sample of N — 145. The frequency distribution of the final 
residuals (of Table C.6) is presented in Table C.12. The mean and standard 
deviation for the residuals are also given in this table. Now the standard 


error of a zero correlation is p 
1 
o = — = .0830. 
727 145 


Evidently the required inequality is satisfied, and the factor pattern may 
be regarded as acceptable. 

Various standards for adequacy of factorization, or "when to stop fac- 
toring,” should yield the same result for a particular problem if they are to 
be equally valid. Of course, if one of the criteria is more crude than the 
others, less emphasis should be placed on the changes which it may indi- 
cate. For the present example this would mean that, if the difference be- 
tween the standard deviation of the final residuals and that of a zero corre- 
lation were considered too large,* the pattern would still not be revised be- 
cause the more exact tests by means of the sampling error formulas indicate 
that the factor pattern is a valid description of the original variables. As a 
matter of fact, the difference 0175 = .0830 — .0655 usually would be con- 
sidered insignificant, so that this check also indicates that the factorization 
has been carried to the proper stage. The important thing to note 1s that 
one cannot merely “turn a crank" and come out with a valid solution, al- 
though according to some crude, cover-all standard it may be deemed ac- 
ceptable. The experimenter should use all the theoretical and factual knowl- 
edge at his disposal at each stage of the analysis to produce the best solution 
from the given techniques and data. 

Before leaving the present example, a word about the naming of factors 
may be in order. It will be recalled that the fundamental purpose of factor 
analysis is to comprehend a large class of phenomena (the values of a set of 
variables) in terms of a small number of concepts (the factors); and for the 
present time, at least, this description is taken to be a linear function of the 
factors. In a mathematical or physical theory it may be sufficient to know 
that twenty-four variables can be described linearly in terms of only six new 
hypothetical ones—that is usually quite an accomplishment, and it is of 
little concern as to what the six new variables are called. But in the biologi- 
cal and social sciences—psychology, for example—it is usually demanded 


and the permissible difference is not known. 


* The standard is only a rough one, 
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that these new variables be named in order that the solution can be given a 
more “practical interpretation." 

The coefficients of a factor pattern indicate the correlations of the vari- 
ables with the respective factors and furnish the basis for naming them. In 
the case of oblique factors, to be discussed in later chapters, the structure 
furnishes the correlations of the variables with the factors, and so it is simi- 
larly employed in naming the factors. The investigator is guided by the 
magnitude of the factor weights in the selection of appropriate names for 
the factors. The name selected is usually suggested by the nature of the 
variables having the largest correlations with the factor under considera- 
tion. This name should be consistent with the nature of the remaining vari- 
ables which have a low correlation with the factor. 

The common factors for the example are named from the pattern given 
in Table 6.8 and the brief descriptions of the tests in Appendix B.1. The 
factor Bo has positive weights throughout and correlates highest with such 
deductive tests as Series Completion (23), Woody-McCall Arithmetic (24), 
Problem Reasoning (22), and Word Classification (8). Hence By might be 
called a “general deductive factor." This name is consistent with the nature 
of the remaining variables—those involving a lesser amount of deductive 
ability have correspondingly smaller factor weights. 

The remaining common factors are named from the subgroups of tests 
which have significant correlations with them. The first group factor is 
named from the “spatial” subgroup (Tests 1-4), the second from the “ver- 
bal” subgroup of tests, and similarly for the remaining factors. The names 
of the six common factors are indicated in Table 6.8. In addition to the 
common factors, there is one unique factor for each of the twenty-four tests. 
If a name were desired for any unique factor, it would be obtained from the 
description of the particular test. The only unnamed factor is the doublet 
D; involved in Speed of Adding (10) and Woody-McCall Arithmetic (24). 
This doublet appears to measure “arithmetical speed," 
as a more significant factor if more tests of this type 
battery to experiment for this purpose. 


For future work with this factor pattern the doublet, will be dropped from 
consideration, since, as was remarked before, it takes at least three vari 
to define a factor. The six common factors may be referred to by means of 
symbols or the descriptive names, which are tentatively assigned for that 
purpose. The particular name by which a factor is designated, however, 
should not raise an issue for dispute. If another investigator chooses to call 
these factors by other names, he is free to do so. The naming of factors is 
not a problem of factor analysis, which is a branch of statistics, but some 


which might appear 
were introduced in a 


ables 
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descriptive names may be highly desirable in a particular subject matter for 
purposes of classification. 


6.11. A Bi-factor Solution for a Reduced Set of Variables 


It should be evident that any factor solution is dependent upon the 
sample of individuals and variables. When either of these is altered, it is to 
be expected that the solution will not remain strictly invariant but may 
exhibit a certain amount of stability. This stability will be illustrated for 


TABLE 6.12 
BI-FACTOR PATTERN FOR THIRTEEN 
PSYCHOLOGICAL TESTS 
Common Factors 
Test Couns 
General | Spatial Percep- | NALITY 
Deduc- Rela- Verbal tual 
tion tions Speed 
j Bo B Bs Bi ^j 
ds s 614 Fo o rmm 558 
2.. 339 4296 Ns ae eases pris isione 203 
3 369 QE rT TE .362 
4 460 390 EEPE TETEN .314 
5 654 ]|.....- 467 |... .646 
(ROn sOUE rri ase .0260 [owe eee 641 
7 578 |... 0645 fon ee eee 750 
8 718 ]|.......- 285 |.......- 571 
9 582 ]|.....-- .689 |... 758 
10.. 944 ]... rne . 703 554 
Ll AO Yas acca win [n rtt a8 .520 449 
12 846 |... fe ero mine .641 531 
13........ e] «40644 eee mmImwÁ .429 599 
Contribution 
offactor..| 3.538 | 0.596 1.441 1.800 j|... 


the case of a fixed sample of individuals and a reduction in the number of 


variables. 

A bi-factor solution for the first thirteen of the preceding set of twenty- 
four psychological tests has been obtained by the methods of this chapter 
and is presented in Table 6.12. The variables of this subset were selected 
80 that the solution should contain three group factors and include only 
about half of the original variables. 

As might be expected, the form 
with the corresponding portion of t 
coefficients for these two patterns are in c 


of solution given in Table 6.12 agrees 
hat given in Table 6.8. The respective 
lose agreement throughout. The 


* * 
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largest discrepancy occurs in the case of the general-factor coefficients for 
variable 10, which have the values .388 and .244, respectively. Inasmuch 
as the standard error of aio, = .388 has already been dibus to be .123, the 
difference .144 is regarded as insignificant. The communalities for corre- 


sponding variables are also in close agreement, the greatest discrepancy 
- being .075 for Test 13. * 


* 
The foregoing illustration is, of course, an example of the stability of one 
type of solution, when the set of variables is reduced. This type of stability 
will also be illustrated for other forms of solution for the same data in Chap- 
-ters X and XI. ty d = ' Y 
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t r CHAPTER VII a 


THE PRINCIPAL-FACTOR SOLUTION 


7.1. Introduction 

The principal-factor method was developed by Hotelling* at the sug- 
gestion of Kelley, who since has developed an alternative procedure, based 
on coyariances instead of correlations. In a still more recent paper Ho- 
tellingt has presented an improvement on his original scheme for calcula- 
tion of principal factors. Hence methods available for calculation include 
Hotelling's first procedure, Kelley's improvement by use of a simple mathe- 
matical device, and Hotelling's latest simplified scheme. For simplicity 
this latter form of computation only is treated here. 

It will be noted that the term “factor” is employed here, as elsewhere in 
the text, rather than “component,” which is used by Hotelling and Kelley. 
This seems advisable especially in later chapters where comparisons of dif- 
ferent solutions are made. Hotelling's objection to the word “factor” arose 
from its possible confusion with the mathematical term, but general usage 
would appear to favor retention of “factor.” * 

As indicated in 2.5 the communalities of the variables are usually the 
desired portions of the variances to be analyzed. In 7.2, therefore, various 
methods for estimating communality are presented. The problem of ob- 
taining these estimates is the same for both the principal-factor and the 
centroid solutions, so the discussion in this section applies also to the meth- 
od treated in Chapter VIII. E UN - 

The essential portions of Hotelling's theoretical treatment of the princi- 
pal-factor solution are given in 7.3, and certain proofs of the fundamental 
development are presented in Appendix B.9. The method of analysis is 


' applied to a matrix of correlations with communalities in the diagonal rather 


than ones or reliabilities as Hotelling suggests. Although he has furnished a 
treatment of the sampling problem, it is not applicable in the present de- 
velopment which involves communalities. When appropriate estimates of 
the communalities are employed, it is shown in 7.2 that the problem of 


"when to stop factoring” is obviated. 
“Analysis of a Complex of Statistical Variables into Principal 


*H; i 
arold Hotelling, , pp. 417-41, 498-520. 


Components," Journal of Educational Psychology, 1933 
T Truman L. Kelley, Essential Traits of Mental Life, 1935. j 
1 “Simplified Calculation of Principal Components," Psychometrika, 1936. 
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Except for the choice of communalities, the solution of a principal-factor 
pattern can be reduced to a routine procedure. An iterative scheme for 
caleulation is described in 7.4, and a detailed series of steps for computation 
is given in Appendix D. In illustrating these steps a set of only eight vari- 
ables is employed inasmuch as this is a sufficient number to clarify the pro- 
cedure. Furthermore, it is usually advisable to restrict the application of 
the direct principal-factor solution to small sets of variables because of the 
excessive amount of calculation when the number is large. In Chapter IX 
a method will be given for obtaining a principal-factor solution by rotation 
from another solution. If the latter method is employed, the labor is greatly 
reduced, and the principal-factor form of solution may thus be made feasible 
for larger sets of variables. 

In section 7.5 the solutions for four sets of variables are presented. The 
first of these analyzes the eight physical variables used for illustration in 
Appendix D. The next application is made for the twenty-four psychologi- 
cal tests of 2.8, in spite of the large amount of calculation entailed. It was 
originally intended to employ this example in the outline of steps in Appen- 
dix D, but because of the bulk of tabular material it was feared that the 
reader would fail to see the trees because of the forest. The detailed com- 
putations of this example are, therefore, not included in the text, but only a 
portion of the pattern is exhibited in 7.6. The third illustration is given 
for eight emotional traits studied by Cyril Burt. The final example is taken 
from the field of political science. A discussion of the factors obtained in 


each example is presented to indicate the effectiveness of the principal- 
factor form of solution. 


7.2. Estimation of Communalities 


It was pointed out in 2.5 that the portions of the variances to be factored 
are determined by the diagonal elements of the correlation matrix. When 
ones are put in the principal diagonal of this matrix, the resulting descrip- 
tions of the n variables are in terms of n (sometimes fewer) common factors. 
This was the approach of Hotelling in his method of analysis. In the present 
treatment, however, communalities are the basic quantities to be analyzed 
in accordance with the assumed composition of variables (2.4). The de- 
velopment of the principal-factor method in the following section is based 
upon the latter assumption of the composition of the variables, and hence 
good estimates of the communalities are required. 

Appropriate communalities, and hence the entire factor pattern, can be 
obtained directly by the methods of Chapter IV if the rank of the correlation 
matrixis suitably approximated. Thusin the bi-factor method the rank is ap- 
proximated by means of the grouping of variables. Then the pattern plan is 
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postulated, and the coefficients are computed directly by formula. If, in- 
stead of approximating the rank, estimates of the communalities are em- 
ployed, it is possible to obtain a solution with a number of factors dependent 
upon such estimates. In the factorial analyses of this and the succeeding 
chapter, the procedure of estimating communalities is generally employed. 

When no such estimates are feasible, owing to the nature and lack of de- 
sign in the set of variables, an arbitrary method will be presented in 8.4 in 
which no explicit use of the diagonal elements is made, and the adequacy of 
the resulting pattern is tested from the final residuals. 

One method for estimating the communality of a variable z; is to select 
the highest correlation among all its correlations with all the other vari- 
ables of the given set. Another method is to employ as an approximation to 
the communality a triad A? = ryatin/Te, where 7; and rj; are the two highest 
correlations for z;. Still another estimate is given by the average of all the 
correlations of each variable. These “arbitrary estimates” of the commu- 
nalities have been employed by workers in multiple-factor analysis, who 
contend that the first method is sufficiently accurate when applied to sets 
of twenty, or more, variables. 

A still more satisfactory procedure for estimating communality can be 
obtained by employing the methods of Chapter IV. By means of the group- 
ing of variables, the approximate rank of the correlation matrix may be 
assumed. In contrast to the bi-factor procedure, however, no pattern plan 
is postulated. The groupings of variables are obtained merely to approxi- 
mate the rank in order to get suitable estimates of the communality. For 
any rank, thus determined, the direct method of obtaining the commu- 
nalities, as described in 4.6, is theoretically most desirable. This method 
involves the calculation of the average of all possible expressions (4.27) for 
each communality, when m is the rank of the correlation matrix. For small 
sets of variables it is expected that there will be only a few factors.* It may 
then be sufficiently accurate to assume rank one or two and employ the 
method of Chapter IV for calculating approximations to the communalities. 
In practice, when the rank exceeds two, the method of Chapter IV is not 
feasible because of the complexity of the formulas and large amount of com- 
putation involved. One simplification of this technique is accomplished by 
Considering the section of the correlation matrix corresponding to a sub- 
group of variables as approximately of rank one. The estimates of the com- 
munalities may then be readily calculated by means of formula (6.9) BD 
plied to the subsets of variables. This procedure seems much more satis- 
factory than the estimation of the communality by a single triad involving 


the two highest correlations of a given 
* See 4.2. 


variable. 


T 
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In order that a subset of variables shall produce a correlation matrix of 
rank one, it is necessary that the vectors representing these variables lie in 
a space of one dimension. Since such a configuration of vectors is not ordi- 
narily to be expected for a complete set of variables, the rank usually will 
be greater than one. An even better estimate of the communality than the 
preceding one may be obtained by selecting appropriate variables out of the 
total set so as to yield a matrix of rank one, and applying the bi-factor tech- 
nique described in 6.4. Although the bi-factor pattern is explicitly obtained, 
it is employed here only as a means of obtaining good estimates of the com- 
munalities to be used in the calculation of other types of factor solutions. In 
general, then, the communalities from any satisfactory solution may be 
taken as good estimates in further factorial analyses. To contrast an arbi- 
trary estimate for a variable involving only a few correlations with that 
based upon the entire correlation matrix, the latter will be designated as a 
“complete estimate” of the communality. 

Inasmuch as the arbitrary estimates may not be close to the desired com- 
munalities, it has been suggested* that the diagonal values be changed in 
each table of residual correlations for the calculation of successive factor 
coefficients. For example, when the highest correlation for each variable 
is taken as its communality, the perplexing problem as to “when to stop 
factoring” arises. In the matrix of residual correlations with the first factor 
removed, the highest value in each column would again be used for the 
diagonal entry (in place of the value actually computed). By continuing 
this process, a large number of factors (relative to the number of variables) 
may be obtained, with some significant coefficients even for the last of many 
factors. 

When the complete estimates of the communalities are employed, the 
foregoing difficulty disappears. Since these estimates are the best available, 
the diagonal values are not altered as in the case of arbitrary estimates. 
Then, in a practical sense, the question of “when to stop factoring” no 
longer arises. The factoring is carried to the stage where nearly 100 per cent 
of the total estimated communality is analyzed. When this is accomplished, 
the number of common factors is relatively small; and, if additional factors 
are obtained, practically all coefficients are insignificant. An illustration of 
this “convergence” to a small number of common factors is given in 8.3, 
where a centroid pattern for the twenty-four psychological tests is obtained. 

The most important advantage in employing the complete estimates of 
the communalities in the calculation of a principal or centroid factor pattern 
arises in connection with the statistical adequacy of these solutions. Such 
estimates are based upon the bi-factor solution, for which sampling error 


* L. L. Thurstone, The Vectors of Mind, 1935, p. 113. 
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formulas are given in Chapter VI. If the bi-factor pattern is justified by the 
various standards, including the sampling formulas, the resulting commu- 
nalities are also validated. Then, by employing these communalities in any 
other form of analysis, further tests of adequacy of solution are obviated. 
From the geometrical point of view, the fundamental problem of factor 
analysis is the determination of the common-factor space. Once this space 
has been determined, any reference system (corresponding to a set of fac- 
tors) can be transformed into another preferred type by means of a suitable 
rotation. In obtaining a bi-factor solution, the common-factor space is 
thereby fixed. Then in further analyses, employing the bi-factor commu- 
nalities, the common-factor space is predetermined. Any factor solution in 
this common-factor space may then be regarded as a linear transformation 
of any other. In actual practice, however, such alternative solutions are 
generally obtained by different statistical procedures. Illustrations of these 


geometric properties are given in Chapter XIII. 


7.3. Principal-Factor Method i 

As indicated in 5.2, when the point representation of a set of variables is 
employed, the loci of uniform frequency density are essentially concentric, 
similar, and similarly situated ellipsoids. The axes of these ellipsoids corre- 
spond to the factors in the principal-factor solution.* From an algebraic 
point of view, the selection of these axes is equivalent to choosing & set of 
factors in decreasing order of their contribution to the total communality. 
The analysis is begun with a factor F, whose contribution to the commu- 
nalities of the variables has as great a total as possible. Then the first- 
factor residual correlations are obtained, including the residual commu- 
nalities. A second factor F», independent of F, with a maximum contribu- 
tion to the residual communality is next found. This process is continued 
until the total communality is analyzed. , 

A brief theoretical aee ohan of the principal-factor method will now be 
given, and this will be followed by & discussion of & more expedient form of 
solution in the next section. If the composition of & statistical variable} is 


again taken to be 
(71 anki 4 aaf: + - -© + Ginllm 
) 2; = aif i2 WR S 


With the unique factor omitted, the communality of z; is then given by 
(7.2) ded broke ce 


* Hotelling, “Analysis of a Complex of Statistical Variables into Principal Com- 
Ponents,” Sec. 3. 


T See n. *, p. 95. 


160 FACTOR ANALYSIS 


In general, the term aj, indicates the contribution of the factor F, to the 
communality of z; The sum of the contributions of the first factor F, to 
the communalities of the » variables is 


(7.3) A,-— a ah +... ah. 


The object of the present method is to choose the coefficients aj so as to 
make A; a maximum, subject to the restrictions that the correlations are 


reproduced by the pattern (7.1). The conditions may be expressed as fol- 
lows: 


(7.4) Tik = Tjk = rasan (d = LQ. ong a 
t=1 


In the succeeding analysis the reproduced correlations rj, will be replaced 
by the corresponding observed correlations rj. As indicated in Chapter VI, 
this procedure implies the assumption of zero residuals. 

By the methods of the calculus it is possible to maximize A; under the 
given conditions.* The resulting system of equations for the solution of the 
unknowns a; may be written as follows: 


(hj — xau + Tiaa + Tista +... + Tinan = 0, 
TQ, + (h? = Naa + Tosa. +... + Tanani = 0 D 
(7.5) Taau + Taaa + (hå — Ajaa +... + Tanan = 0, 


Tmi + T2021 + Tnađai +... + (h2 — 3)a1 = 0, 
where A is a parameter independent of the a’s. A necessary and sufficient 
condition for the system of equations (7.5) to have a solution (in which not 


all unknowns are zero) is the vanishing of the determinant of the coeffi- 
cients} of the aj. This condition may be written in the form 


(—» nm 


Tia eee Tin 
Tar (hg — N) ra Ton 
(7.6) Ta Taz (h$ — 2) Tan =0 
Tr Tro Tna see (R-N) 


* For a proof see Appen. B.9. 
1 L. E. Dickson, Modern Algebraic Theories, 1930, p. 61. 
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An equation of the form (7.6) is known as a characteristic equation, some of 
its properties being that all the roots are real and that a g-fold multiple root 
substituted for A in (7.6) reduces the rank of the determinant to (n — q). 

When a simple root of the characteristic equation is substituted for ^ in 
(7.5), a set of homogeneous linear equations of rank (n — 1) is obtained. 
This set of equations has a family of solutions, all of which are proportional 
to one particular solution. It follows* from the analysis for maximizing Ai 


that the factor of proportionality is X = Ma = A,. Hence Ay, which is 


j=l zan y 
to be maximized, is equal to one of the roots of the characteristic equation, 


namely, the largest root X. 

The problem of finding the coefficients aj of the first factor Fi, which 
will account for as much of the total communality as possible, is then solved. 
The largest root X; of (7.6) is substituted into (7.5), and any solution an, 
aa, .. . , am ls obtained. Then, to satisfy the relation (7.3), these values are 
divided by the square root of the sum of their squares and then multiplied 


by V^. The resulting quantities are 

ment um LBs ns 
PU Mal, + al eso 08 

the factor pattern (7.1). It may 


fy the condition (7.3); for, upon 
there results 


(7.7) 


which are the desired coefficients of Fi in 
be observed that these values of aj: satis 
squaring the expressions (7.7) and summing, 


wn th) hy Ay 


sa _ (ah + oh + a 
= si aj + eh +--+ + om 

When the largest root of the characteristic equation is a g-fold multiple 
root (i.e., the q largest roots are equal to Xj), & set of n homogeneous linear 
equations of rank (n — q) is obtained upon substituting this value for din 
(7.5). This system of equations has q linearly independent solutions, while 
every other solution is linearly dependent on them. Designating these solu- 


tions b 
y 
au, Ga, «+? Anl; 
dig, G22, c5? n2 ; 
Qu, Gar ccc? Ang s 


* See Appen. B.9, eq. (B.28). 
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they may be selected so that they are “orthogonal” to each other, in the 
sense that 


n 
> aj, Ai 


(78) j=1 (^ bad, RET ? ; 


n 
> a;,a;, = 0 


j=1 


The q sets of solutions may then be taken as the coefficients of g independ- 
ent factors Fi, F2, . . . , F, which account (equally) for as large as possible 
a part of the total communality. Inasmuch as this case is very unlikely to 
arise in actual practice, it will not be treated explicitly in the subsequent 
analysis. 

Having determined the coefficients aj of the first factor F, the next prob- 
lem is to find a factor which will account for a maximum of the residual 
communality. In order to do this, it is necessary to obtain the first-factor 
residual correlations. Furthermore, in obtaining still other factors the resid- 
ual correlations with two, three, ..., (m — 1) factors removed are em- 
ployed, and hence a suitable notation is required. In the last chapter only 
residuals with the general factor removed (7) and final residuals (7) were 
explicitly indicated. A convenient notation for the residual correlation of 
Tix With p factors removed is pr;,. Thus, when the first factor has been ob- 
tained, the first-factor residuals are written in the form 


(7.9) Wik = Tik — Apia = Ajzak + Ajaks +... + Gina » 


In determining the coefficients of the second factor Fs, it is necessary to 
maximize the quantity 


(7.10) Ay = di + ay +... + al, 


which is the sum of the contributions of F, to the residual communality. 
This maximization is subject to the conditions (7.9), which is analogous to 
the restrictions (7.4) in the case of the first factor. The analysis for obtain- 
ing the coefficients aj; is parallel to that for determining the first-factor 
coefficients, the only difference being that the residuals 17; are used in place 
of the correlations r;,. Proceeding in this way, the coefficients of Fils, 
..., Fm are determined in the order of their contributions to the total 
communality. 

An important mathematical property of the principal-factor pattern is 
that of orthogonality. It has been indicated already that in case the largest 
root of the characteristic equation is a multiple root, this orthogonality is 
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given by (7.8). The property of orthogonality holds, in general, for the 
coefficients of all the principal factors. 'These relations may be written as 
follows: 


B =k 
j=1 p TA 


n ssl 
DY asaj =0 
j=1 


The properties (7.11) are useful in checking the numerical calculations of 
the factor coefficients. 

The foregoing description 
sidered as its logical basis but 
direct solution of the characteristic equation & 
equations such as (7.5) would involve great a 
reason, an iterative method which requires only r 
be described in the next section. 


(7.11) 


of the principal-factor method may be con- 
not as a scheme for actual computation. The 
nd sets of linear homogeneous 
Igebraic difficulties. For this 
outine computations will 


7.4. Iterative Form of Solution 

A simplified method for calculating a principal-factor pattern will now 
be developed. This treatment is based upon the two fundamental papers of 
Hotelling in which he first presents the iterative scheme and later gives an 
improved variation of it. Detailed steps for the numerical calculation of the 
pattern are given in Appendix D. . 

The iterative form of solution yields a root of the characteristic equation 
and the corresponding coefficients in the factor pattern simultaneously. 
The roots appear in descending order of magnitude upon successive applica- 
tions of the iterative scheme. When complete estimates of the commu- 
nalities are employed, the number of roots required to account for this total 
communality is relatively small. The contribution of any factor, say, after 
95 per cent of the communality has been analyzed, would be expected to be 
insignificant. The iterative procedure, based upon complete estimates of 
the communalities, then affords an adequate factorization. 

In the iterative process an arbitrary set of n numbers is selected, and, 
after manipulating these numbers with the matrix of correlations, they are 


finally reduced to the desired coefficients of the first principal factor. Thus, 
the arbitrary set and consider the 


take the numbers an, az - ++» @m 4S 
following transformation 
n 
(7.12) aj > Tika G= 120659) 


k=1 
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to the new set (a, a4, .. . , ahı). If the numbers aj are proportional to 
the direction cosines of any line through the origin, then the numbers aj, are 
proportional to the direction cosines of a new line (through the origin) cor- 
responding to the original line under the rotation (7.12). 

In general, the line associated with the numbers a; is distinct from the 
line corresponding to the aj. There may, however, be lines which remain 
fixed under the transformation (7.12). Such invariant lines are those for 
which a quantity à exists such that 


(7.13) af, = Aag (212...,9. 


Upon substituting the values (7.13) in (7.12), that expression reduces to 


Aag = 
k 


Tikar = Tan + Tjaa +... + APay +... + Tinan 
1 


n 


or 
(7.14) Taan + ryan +... + (RF — Naat... +rinam = 0. 


As j takes on the values 1 to n in (7.14), it is readily seen that these equa- 
tions are identical with (7.5). Thus, for any invariant line, the direction 
cosines are proportional to a solution of (7.5), where is a root of the charac- 
teristic equation (7.6). Hence it follows that the invariant lines are the de- 
sired principal axes. It is thus apparent that, if a set of numbers an, a2 

- ; amı can be found which when substituted into the right-hand members 
of (7.12) produce relations (7.13), the numbers a/,, aj,,..., ah are pro- 
portional to the direction cosines of the principal axes. The coefficients of 
one of the principal factors can then be obtained from the latter set of num- 
bers. Furthermore, ^ in (7.13) is the sum of the contributions of this factor 
to the communalities of the variables. 

In practice, of course, it cannot be expected that the arbitrary numbers 
aj1 will be so selected as to be proportional to the direction cosines of one 
of the principal axes. The iterative process then involves the use of the 
derived numbers oj, as a new set of arbitrary numbers in place of aj. Now 
if the numbers aj, are proportional to the direction cosines of any line 
through the origin, and if they are substituted in the equations* 


n 
(7.18) aj = > Taah SB sey BG 
j=1 

* The symbol asi is employed instead of a; corresponding to the notation in (7.12) 


because it will be found convenient for subse i i inj 
it w | quent algeb i sa 
a subscript in the right-hand member of (7.15). gn USE 
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then the quantities on the left will be proportional to the direction cosines 
of a new line (through the origin) corresponding to the original one under 
the transformation (7.15). This process is continued until the ratios among 
the quantities obtained at any stage converge to the corresponding ratios 
among the coefficients of / to any specified degree of accuracy. The proof 
of the convergence of these ratios to those of the coefficients aj; of the first 
principal factor is given by Hotelling.* A convenient procedure is to divide 
each of the trial values by a fixed one of them, say the largest. Then the 
next value obtained, corresponding to this number, will be an approxima- 
tion to the characteristic root X. 
Instead of calculating the successiv: 
them in equations like (7.12) and (7.1 
duced which greatly accelerates conv 
complished by the formal substitution 0: 
upon substituting the values for aj, from 
of (7.15), the latter equation takes the form r 


n n 
ay = ` > TojV jk - 


k=1 j=1 


e values aji, avi, etc., and substituting 
5), a modification will next be intro- 
ergence.] This simplification is ac- 
f the a’s in these equations. Thus, 
(7.12) into the right-hand member 


This expression may be written more simply as follows: 


(7.16) aj = > coan (/21,2,...,n), 
kel 
where 
n 
(7.17) Cor = XC i 
j=l 


Consequently, if the sums of the paired products Cox are first obtained, equa- 
tion (7.16) may be used instead of (7.12) to calculate us abi values. Hence 
the two iterations which were involved in obtaining a» (by multiplication 
with the correlations) are reduced to only one operation with the num- 
bers c,,. The number of iterations required for any degree of accuracy is 


thus cut in half. 


It will be found very convenient to employ matrix notation in dealing 


with complex expressions of the form (1.17). ER = lir;zl| is used to denote 
the matrix of correlations, the Cvk is the element in the vth row and kth 
column of the product of the symmetric matrix R by itself. The matrix 
bles into Principal Components," Sec. 4. 


* "Analysis of a Complex of Statistical Varia : 
pal Components. 2 


t Hotelling, “Simplified Calculation of Princi 
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of these elements may be represented by C = |jc,x||. It is, therefore, ap- 
parent that 


C = |levall = (iral - lr] = R'* R= R*, 


where the last equality follows from the fact that the transpose of a sym- 
metric matrix is equal to the matrix itself. Substitution of the trial values 
an, azn .. . , 4m in (7.12) is equivalent to multiplying them by R (i.e. 
obtaining the sums of the paired products with the elements of the col- 
umns of R). Substitution in (7.16), on the other hand, amounts to the mul- 
tiplication by R?. 

The improvement in the iteration process need not end with the employ- 
ment of R?. After doubling the speed of convergence by squaring R, it can 
be doubled again by squaring R?, i.e., by multiplying a set of trial values 
by R‘, and thus the equivalent of four multiplications by R is obtained. 
Upon squaring again, a matrix Rê is obtained, and multiplication by it is 
equivalent to eight multiplications by R, and so forth to any power of the 
correlation matrix. This squaring process is continued until the conver- 
gence is so rapid that additional matrix squaring is not worth while. In 
Appendix D a scheme for determining the number of times a matrix should 
be squared is explained. 

The second and remaining principal factors may be determined by the 
same method, and the convergence can be accelerated by the use of a con- 
venient power of the matrix of residual correlations. It isnot necessary, how- 
ever, to obtain this power of the residual matrix by repeated squarings, 28 
was done in the case of the original matrix of correlations. Instead, the de- 
termination already made of the power of R and the following algebraic 
properties of matrices can be employed for this purpose. 

Denoting the matrix of first-factor residuals by Ru, it follows from (7.9) 
that 
(7.18) Ri = [ral = lri — annal = R — Qi, 
where 


Qi = [pill = lanl] —Gk-12.... 


is used to represent the matrix of products of first-factor coefficients. Now 
it is shown in equation (B.30) that 


` Tiji = Man 


and in (7.11) that 


— 
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These lead to the following relationships among the matrices:* 


— [8s OR =n 


By repeated use of (7.19) it can readily be shown that 


(7.20) a^ : cR MQ, 


where e is any positive integer. Hence employing (7.19), 


R? = (R — Qi)? = R? - 2RQ: + Qf = RM, 


and, in general, 
(7.21) Ri = R: P MQ, E 


x is expressed in terms of the eth 


Thus the eth power of the residual matri 
obviating actual squaring of the 


power of the original correlation matrix, 
residual matrix. 
From the foregoing development 


Scheme may be summarized. Using R' 
of trial values, this set rapidly yields the values of the first-factor coeffi- 


cientst and the characteristic root X. Furthermore, the value M will be 
determined from the multiplication of the set of trial values by R*, and 
A! can be obtained by divison. Then multiplying M7! by each element 
of Qı and subtracting from the corresponding element of R*, the eth power 
of the residual matrix is obtained. The second-factor coefficients are ob- 


the algebra leading to the last relation will be 
= |janarl] is given by 
Gj, Ev-21,2,....9, 


the order of procedure of the iterative 
e as the basis of selection of the set 


_ * To indicate the derivation of (7.19), 
given in detail. The square of the matrix Qı 
Qe jaja asia 

tation of Q: and the kth column of the second 
representation are obtained by letting v range from 1 to n. Thus a representative row is 
lana, apan . . . ajanı] and a representative column (written horizontally to save space) 
is {anar anar . . « Andes} Row-by-column multiplication of the matrices then pro- 


duces 
2 
[01 > ayy 


zn 


in which the jth row of the first represen 


= |lanaeAal| = MQ: - 


Qi = | ananas 
ET 
es" (Psychometrika, 1938), Burt has pointed 
out that to factor a matrix R* is equivalent to obtaining & Spearman general factor. 
This arises from the fact that, with & sufficient number of self-multiplications, any sym- 


metric matrix can be reduced as closely 85 desired to a matrix of rank one. 


t In “The Unit Hierarchy and Its Properti 


168 FACTOR ANALYSIS 


tained from R; and R1 in the same manner as the first-factor coefficients are 
determined from R and Re, It may not be necessary to employ the eth 
power of R; in the calculation of the second-factor coefficients when rapid 
convergence is evident. Then some lower power of Ri, or R; itself, is em- 
ployed. This is illustrated in Appendix D. To calculate the third-factor 
coefficients, the matrix Rs of the second-factor residuals and R$ (or some 
lower power of R+) are employed. The latter matrix is obtained convenient- 
ly by an expression of the form (7.21) relating the second- to the first-factor 
residuals. Further factors are determined similarly until approximately all 
the communality is analyzed. 


7.5. Numerical Applications 


1. EIGHT PHYSICAL VARIABLES 


The first illustration for the method of principal factors was obtained 
from Mrs. Mullen’s data, which were referred to in 4.6. Her analysis of 
seventeen physical variables by the bi-factor method revealed a general 
size factor and two outstanding group factors identified with longitudinal 
and horizontal growth. After this solution was obtained, an alternative 
form, which might be preferred by the biologist, was suggested by the fact 
that the latter two factors might be considered as opposing measures of a 
single factor. A solution with a general physical growth factor and a bipolar 
factor* (representing the longitudinal versus horizontal growth) was then 
proposed, 

For the purpose of making such a principal-factor solution eight of the 
total set of variables were selected. Of the set of seventeen variables, three 
indices—of head size, hand squeeze, and lung capacity—were eliminated be- 
cause they were not measures of the two group factors. The following vari- 
ables were dropped because of small, or insignificant, group-factor coeffi- 
cients: sitting height, bi-iliac diameter, chest depth, and shoulder width. 
The eight variables retained include two subgroups consisting of four longi- 
tudinal and four horizontal variables. The correlations among these vari- 
ables are presented in Table 7.1. 

In Mrs. Mullen’s study the complete analysis was made by the bi-factor 
method which furnished the communalities for all variables. The portion 
of the bi-factor pattern for the eight variables, also including their commu- 
nalities, is given in Table 7.2. Thus estimates of the communalities of these 
variables for further analysis into a principal-factor pattern are immediately 
available. The communalities of these variables were also obtained directly 
from the correlations by the methods of Chapter IV. The latter values, 

* See 5.7. 


THE PRINCIPAL-FACTOR SOLUTION 169 


ra are given in Table 4.5, are compared with the bi-factor estimates of 
he communalities in Table 7.3. It is evident from this comparison that 
: ere would be little practical difference between the final solution based 
— IT m sets of communalities. Since the communalities of Table 7.2 
re available at the time the analysis was first made, they are e j 
i r mpl 
Appendix D. iin — 
TABLE 7.1 


INTERCORRELATIONS OF EIGHT PHYSICAL VARIABLES 
FOR 305 FIFTEEN- YEAR-OLD GIRLS 


Variable | 1 | 2 | 3 | 4 5 6 7 8 
A, -Hrelplibuss sss ways 
Wc —! ldem 
2 Length of forearm. 805 | .881 |.....-|-- 
H ength of lower leg. .859 | .826 | .801 |....- 
A Weight. iu crass sees cem ‘473 | -376 | . ABB: |p asc 
T Eitrodhantéro diameter.| .398 | .326 | . E R - 
& e est girth cwn sd EN .301 | .277 | .237 R «DB. esas 
e Chest width...........- 7382 | .415 | .345 | .365 .629 | .577 


TABLE 7.2 
PORTION OF BI-FACTOR PATTERN FOR SEVENTEEN 
PHYSICAL VARIABLES" 


General Caines 

Variable Physical Lankiness | Stockiness nalit 
Growth y 

j A B (d È 
1 .691 .854 
2.. .591 .897 
3.. .581 .833 
4: .598 .783 
5.. .694 .870 
6.. .611 .687 
Ms .562 -521 
8 .596 .579 


"Factors in the Growth of Girls Seven to 


a Tak F Mullen, 
aken from Frances Die Department of Education, Uni- 


Seventeen Years of Age" (Ph.D. dissertation, 

versity of Chicago), 1939, Table 27, p. 45. 
The principal-factor pattern for the eight variables is presented in Table 
7.4. The adequacy of factorization is shown in several ways. The compari- 
son of communalities in the last column of Table 7.4 shows that the values 
obtained from the pattern differ only slightly from those originally em- 
Ployed in the correlation matrix. ‘Actually, the total calculated commu- 
nality (5.968) accounts for 99.1 per cent of the original communality (6.024). 
he completeness of factorization is also indicated in the process of com- 
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puting the respective factor coefficients. Thus the contribution of the first 
factor (i.e., the sum of the squares of its coefficients) is 4.455 or 74.0 per cent 
of the original communality. Similarly, the contribution of the second fac- 


TABLE 7.3 


COMPARISON OF COMMUNALITIES FOR 
EIGHT PHYSICAL VARIABLES 


| 
(1) (2) r 
Variable Calculated | Bi-factor | ps 
Directly Estimates | ^ 
.842 .854 | —.012 
.881 .897 —.016 
.817 .833 —.016 
.815 .783 .032 
.872 .870 .002 
.647 .687 —.040 
.584 -921 .063 
-502 .579 —.077 
TABLE 7.4 


PRINCIPAL-FACTOR PATTERN FOR EIGHT 
PHYSICAL VARIABLES 


PATTERN Corrricrents® COoMMUNALITY 


VARIABLE ið 

" a) (2) = 
j i d Ui Original |Calculated ai= 
J ESQ: ue usano 2c unan .858 | —.328 | .395 .854 .844 .010 
2. Arm span......... .849 | —.414 | 1328 .897 .892 . 005 
3. Length of forearm. 810 | —.412| 1417 833 .826 007 
4. Length of lower leg. 825 | —.339 | (452 783 | .796 | —.013 
DUBVCIERE. usi een eius 747 -561 357 870 .873 | —.003 
6. Bitrochanterie diameter..| .637 -507 | .581 | .687 | 1663 .024 
7. Chest girth. ............ .561 -488 | .669 | :521 -553 | —.032 
8. Chest width. ss orvs i .619 -871 | .692 .579 521 058 
5.968 054 
Contribution of factor...... 4.455 DBM Wie se ice] uas aeltez oo calles con cur 

Per cent of total original OMe Wee. fo p 498 MHpSRA HE SER Y 
murality:i.. ora euis 74.0 Z5 Nn Dee ee an 99.1 9 


Since the reliubility of any one of these physical variables į i i in each 

li a phys is close to unity, tor in ear 
i then approximate 1) sieentally the specific factor. The index of «sem tae Che unique factor in each 
is then approximately 100 times the calculated communality of each variable 


tor is 1.511 or 25.1 per cent of the co 
factors are required to account for p 
ance. Completeness of factorizatio 
residuals in Appendix D, 


mmunality. In this example only two 
practically all the common-factor vari- 
n is also evident from Table D.13 of final 
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, The coefficients of the first factor in Table 7.4 are all large and positive, 
indicating an important general factor of physical growth (G) among these 
variables. On the other hand, the second factor has loadings of opposite 
signs for the two subgroups of variables. From the nature of the variables, 
this bipolar factor might be called “Stockiness.” If desired, of course, the 
signs of all the coefficients of this factor may be changed. Then this factor 
might be labeled *Lankiness." 

Whatever name is selected for a bipolar factor, it should have a clearly 
recognizable negation. À more fundamental approach is to find a basic 
term which connotes the entire continuum. For example, a bipolar factor 
which is named “Heat” (or, Cold") would have the opposite characteristic 
“Cold” (or, Heat”). A name representing both of these characteristics is 
“Temperature.” These two approaches may be indicated schematically as 
in Figure 7.1. Another example is a bipolar factor named “Fear,” the nega- 


a) Cold Heat 


0 


b) b ae 
Temperature 


0 


Fio. 7.1 


tive of which is clearly Courage." A fundamental term to describe this 
continuum, however, is difficult to determine. The investigator may have 
trouble in finding an appropriate name, of either type, for a bipolar factor 
because of the psychological difficulty of projecting the interpretation be- 
yond the immediate content of the data. 

Inasmuch as *Stockiness" and "Tankiness'" are not clearly distinguish- 
able as opposites (according to a of Fig. 7.1), neither of these seems to be 
an appropriate name for the bipolar factor. In an attempt to get a name, of 
the type b, which transcends the specific descriptions of the variables, the 
term “Body Type" (BT) has been adopted. On this continuum, variables 
describing different body types have projections of opposite sign. — 

Geometric interpretations of the two factor solutions will now be given. 
In Figure 7.2 the coordinates of the points representing the eight variables, 
from Table 7.2, are plotted with respect to the three bi-factor axes. Thus 
the coordinates of the first point are (.691, .614, 0). It will be noted that all 
Points have a zero coordinate for either B or C, and hence lie in one of the 


reference planes determined by the A and B axes or the A and C axes. It 
Should be observed that the points lie exactly in the reference planes only 
in the sense of (2.4). The pro- 


because they represent theoretical variables, i 
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jections of the first four points on the A and B axes are indicated by crosses, 
and the projections of the last four points on the A and C axes are given 
by circles. The cluster of positive projections on the A axis is the geometric 
basis for the naming of the general factor, while the clusters on the other 
two axes furnish interpretations for the group factors. 

The diagram for these eight variables in the plane of the two principal 
factors is presented in Figure 7.3, the coordinates being taken from Table 
7.4. The two subgroups of variables lie in the first and fourth quadrants. 


| 
| 


x 1 
4 

ÉL fs 
B 2 


Fia. 7.2 


Hence the projections of all the points form a single cluster on the positive 
end of the G axis. The projections on the BT axis, on the other hand, fall 
into two clusters which are widely separated. The projections on the re- 
spective axes give the geometric basis for the naming of the general and bi- 
polar factors. 

The bi-factor analysis of the eight variables, as shown in Table 7.2, in- 
dicates a general physical growth factor A, a lankiness factor B, and a 
stockiness factor C. Such a solution is certainly a satisfactory description 
of the variables in terms of factors with positive coefficients. For these 
physical variables, however, the biologist might prefer an analysis in terms 


THE PRINCIPAL-FACTOR SOLUTION 173 


of a general factor and another factor which expresses the two preceding 
factors simultaneously, as indicated in Figure 7.3. This preference might 
be based upon the opposing nature of the two groups of variables which 
makes it possible to conceive of them as measures in opposite directions on 
a common scale. In this example such preference would also lead to parsi- 
inasmuch as the total number of common factors is 


mony of description, 
logist would then choose the principal- 


reduced from three to two. The bio 
factor solution of Table 7.4. 


BT 


Fia. 7.3 


These considerations as to choice of form of solution are in harmony with 


the principles set down in Chapter V. One of the most important bases for 
the selection of a preferred type of solution in any field of investigation Is 
the nature of the variables. Statistical standards, such as complexity of 
variables or parsimony of factors, are of lesser importance in making y; € 
Cision. The investigator is fortunate when the variables of his study len 

themselves to a clear-cut choice of solution. In such a case, the final pattern 
could be considered almost unique, in the sense that other workers would 


also accept it as the preferred type. 
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2. TWENTY-FOUR PSYCHOLOGICAL TESTS 

The next illustration is based upon the twenty-four psychological tests 
of 2.8. Although it would not be expected that variables of this nature 
would ordinarily be interpreted in bipolar form by psychologists, it was de- 
cided to carry the analysis far enough to see the essence of a principal-factor 


TABLE 7.5 


First Two PRINCIPAL FACTORS FOR 
TWENTY-FOUR PSYCHO- 
LOGICAL TESTS 


Variable G VR 


T d gd NE b qb 
r 2i 

3 

N 


I 
N 
= 
o 


Per cent of total 
original com- 
munality....| 65.4 14.3 


solution. This implied the calculation of the coefficients for only the first 
two factors, for all the other factors would be of the form of the second. 
"Table 7.5 presents this portion of the principal-factor pattern. 

The labor of computation in the direct principal-factor solution increases 
very rapidly with the number of variables and factors. For the example of 
eight physical variables, the total time for complete factorization was less 
than ten hours, indicating the feasibility of this method for such set of 
variables. In the case of the twenty-four psychological tests, however, the 


| 
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time required for the calculation of the first-factor weights alone was more 
than seventy hours. Each additional factor would probably require up- 
ward of forty hours depending upon the number of iterations. In e 
machine is devised which will simplify the type of multiplication involved 
the direct principal-factor method will then be appropriate for large sets of 
variables. 
The interpretation of the two factors in Table 7.5 will now be given. As in 
all cases of principal-factor patterns, based upon a set of positive correla- 
tions, the first factor has appreciably high positive loadings for every test. 
This factor might then be regarded as general ability (G). 
: The second factor may be named from the tests with significant coeffi- 
cients. As a rough estimate the standard error of a factor coefficient may 
be taken from Table H.1. For the present example this approximation to 


the standard error* is .109 so that the level of significance of .30 may be 


appropriate, Those variables which have significant coefficients, by this 


standard, are listed below: 


Variable Ccefficient 

5. General information....... 00 .921 

6. Paragraph comprehension.......- 5 .402 

7. Sentence completion....... 5500007 .449 

9. Word meaning... 7 .449 

10. Speed of addition... onn — .430 
11. Speed of code translation...... «s —.355 
12. Speed of counting dots... —.492 
— .348 


18. Number-figure memory... 5000007 

: erbal in character have positive correlations 
with the second factor, while the last four, which are essentially speed tests, 
have negative correlations with this factor. These subgroups of tests agree 
substantially with the grouping by the method of B-coefficients described 
in 2.9. (Although Test 18 was put in the memory group, it is also a meas- 
ure of speed of reaction.) It appears difficult to find the common element 
which underlies the content of these eight variables by which to name the 
factor. A tentative name which might be attached to this factor is “Verbal 


Rigidity” (VR). 

. Subsequent factors will, of course, be of the bipolar form and might be 
given similar interpretations in terms of the subgroups of tests with signifi- 
cant loadings. Such subgroups may include some variables which were em- 
ployed in the naming of VR. Owing to this increased complexity of the 
Variables, the naming of later factors may be more involved. A complete 
derived principal-factor pattern for these twenty-four tests is given in 9.3. 


* See p. 149. 


The first four tests, which are v 
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3. EIGHT EMOTIONAL TRAITS 


The particular appropriateness of the principal-factor method to another 
field of psychology will now be indicated. The material for this illustration 
includes eight emotional traits, the correlations for which were furnished 


TABLE 7.6 


INTERCORRELATIONS OF EIGHT EMOTIONAL VARIABLES 
FOR 172 NORMAL CHILDREN AGED NINE TO TWELVE 


Variable 
1. Sociability..... 
2. Sorrow........ 
3. Tenderness.... 
A. JOVE S sesat cms 
5. Wonder....... 
6. Disgust... 
7. Anger..... 
8, Bean... va es 
TABLE 7.7 
PRINCIPAL-FACTOR PATTERN FOR EIGHT EMOTIONAL TRAITS 
Common Factors CoxMUNALITY 
VARIABLE 
a) (2) 
es d Original |Caleulated| ()—(?) 
1. Sociability... .98 .06 .94 .96 —.02 
2. Sorrow...... .95 —.14 .94 .92 .02 
3. Tenderness... .81 —.51 89 92 —.03 
4, JOY iasa .72 —.10 50 53 —.08 
5. Wonder..... .68 .32 57 56 01 
6. Disgust...... .53 .14 28 30 —.02 
7. Anger....... 52 .60 63 63 00 
[- 0 CAR ABMS 35 —.14 12 14 —.02 
SEOUAL: sepulti Leia SOR wal Sti Reds 4.87 4.96 —.09 
Contribution of| 
factor........ 4.17 SO [espe ono | coin sene i| seras te 
Per cent of total 
original com- 
munality..... 85.6 16:2. saswescien 101.8 —1.8 


by Burt,* who has also discussed the factorial analyses of variables of this 
type in & subsequent paper.[ Although he does not furnish & principal- 
factor solution, he does point out the bipolar nature of emotionality factors. 

* Cyril Burt, “General and Specific Factors Underlying the Primary Emotions," Re- 
port of the British, Association for the Advancement of Science, 1915, pp. 694-96. 


t “The Factorial Analysis of Emotional Traits,” Character and Personality, 1939, pP- 
238-54, 285-99. 
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The traits which are analyzed here are indicated in Table 7.6, where the 
correlations are presented. The communalities were determined from a solu- 
tion essentially of the bi-factor form. It was found that only two principal 
factors were necessary to account for these communalities. The factor pat- 
tern is given in Table 7.7, where only the common factors are presented. 

The first factor may appropriately be called “General Emotionality” 
(GE), although the present sample of emotional traits is small. In the pres- 
ent example the method of naming the bipolar factor is similar to that em- 
ployed in the case of the twenty-four tests. An approximate measure of the 
standard error of a factor coefficient for N = 172 and an average correla- 
tion of .48 is .065 from Table H.1. The traits with significant coeffi- 
cients are anger (.60), wonder (.32), and tenderness (—.51). Since wonder 
and anger are indicative of an egocentric personality, and tenderness is 
indicative of timidity, the factor characterizing these two opposing emo- 
tions may be called “Bgocentricity” (E). If it is desired to change the signs 
of all the coefficients, then the factor may be called “Timidity.” In Burt’s 
discussion fear and sorrow are classed with tenderness, and in the present 
analysis each of these traits has a coefficient of —.14. These values have 
some statistical significance and help substantiate the naming of the sec- 
ond factor. 

4, EIGHT POLITICAL VARIABLES 

The final example has been selected from a set of political variables in 
order to illustrate the applicability of the principal-factor solution in an 
entirely different field. The data also furnish a solution in which all the 
factors, including the first, are of the bipolar form. A set of eight variables 
was selected from a larger group of seventeen political variables, analyzed 
by Gosnell and Schmidt.* The smaller set, which was taken for simplicity, 
nevertheless includes the variables which are among the best measures of the 
factors given in Gosnell's solution. A brief description of these variables, 
measured in 147 Chicago election areas, follows: 

- Lewis: Percentage of the total Democratic and Republican vote cast for Lewis 

. Roosevelt: Corresponding percentage for Roosevelt 

- Party voting: Percentage that the straight-party vote: 

- Median rental: Median rental (in dollars) —— : 

- Homeownership: Percentage of the total families that own their homes 

1 Unemployment: Percentage unemployed in 1921 of the gainful workers ten years 

Oi 

7. M. "o mpm of total families that have lived less than one year at pres- 
ent address 

8. Education: Percentage of population, 
more than ten grades of school 

* Harold F. Gosnell and Margaret Schmidt, 
the1934 Vote in Chicago,” Journal of the American Stat 


s were of the total 


onRwNnEe 


eighteen years and older, which completed 


“Factorial and Correlational Analysis of 
istical Association, 1936, pp. 507-18. 
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The intercorrelations of these variables are given in Table 7.8, in which 
communalities are also recorded. These communalities were computed by 
the second method of 4.6, in which grouping of variables is not employed. 
For example, the calculation of hj was based upon the stub of Table 4.6. 
Of the fourteen evaluations of this communality, four were dropped because 
they involved insignificant denominators. The remaining ten (.57, .42, .52, 
.55, .61, .39, .66, .45, .47, and .58) were averaged to obtain hj = .52. 

The principal-factor pattern is presented in Table 7.9. It may be ob- 
served that several variables have large negative coefficients for the first 
factor, in contrast with the consistently positive coefficients found in the 


TABLE 7.8 


INTERCORRELATIONS OF EIGHT POLITICAL 
VARIABLES FOR 147 ELECTION AREAS 


Roose- Party Median d Unem- Mo- Educa- 
Variable velt Voting Rental ployment | bility tion 


preceding solutions. In the present solution the first factor, being of the 
bipolar type, may be named from the nature of the variables in the subsets 
(1, 2, 3) and (4, 7, 8). The variables of the first subset may be regarded as 
measures of the “Traditional Democratic Vote" (TDV), which is taken 8$ 
the name of the factor. The variables of the second subset characterize the 
sociological level of the election areas and seem to be opposite in nature 
to the “Traditional Democratic Vote." The high weight for variable 6 (Un- 
employment) is consistent with the foregoing interpretation inasmuch 85 
high unemployment is associated with traditional vote. 

In the case of the second factor, the largest weights appear for variables 
5 (Homeownership) and 7 (Mobility), being 4-.65 and —.56, respectively. 
Inasmuch as both Homeownership and lack of Mobility are aspects of & 
single characteristic, the second factor may be termed “Home Permanency K 
(HP). The negative factor weights for the first three variables again appear 
to verify the naming of this factor. This bipolar factor is conveniently de- 
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. scribed by a single name because the opposing variables may be considered 


as measures on a single scale in opposite directions. 
From the foregoing illustrations it is apparent that the direct principal- 
factor solution is well adapted to small sets of variables, say, less than 


TABLE 7.9 
PRINCIPAL-FACTOR PATTERN FOR EIGHT POLITICAL VARIABLES 


Common Factors COMMUNALITY * 
VARIABLE 
a) (2) " 
j TDY EE Oilear |Csledaud| VA 
l. 52 55 | —.08 
2. 1:00 1.00 00 
3. 78 ‘79 | —.01 
4. 82 -78 04 
5. 36 50 | —.14 
6. 80 .70 01 
7. 63 SS | c3 
8 97 .94 03 
5.88 6.10 | —.22 
Contribution of | 
factors « «i22 5.01 140 li. [erm ntm 
Per cent of total 
original com-| | 
munality..... 85.2 188 eee 108.7 | -3.7 
1 I 


twelve. The direct method does not appear to be feasible when dealing 
With larger sets of variables because of the excessive labor involved. In 
Chapter IX, however, an indirect method is presented for obtaining a 


Principal-factor pattern by rotation from some other form of solution. Thus 
‘ined, it may be rotated to the 


if Some factor solution can be simply obtaine í 
Principal-factor form. By this procedure & large set of variables can be 
analyzed in the principal-factor form with only a fraction of the labor re- 


Quired by the direct method. 


CHAPTER VIII 
THE CENTROID SOLUTION 


8.1. Introduction 


In the two preceding chapters methods have been presented for obtaining 
two of the preferred types of factor solutions discussed in Chapter V. À 
third type of preferred pattern, the multiple-factor solution, was indicated 
in 5.6. A direct method of analysis for this type of pattern is not available. 
Instead, any factor pattern which has the property (2.24) of reproducing the 
correlations may first be obtained and then transformed into the multiple- 
factor form by the methods of Chapter X. The problem of obtaining a mul- 
tiple-factor pattern thus consists of two parts: first, the calculation of & 
preliminary pattern and, second, the transformation of this pattern to the 
multiple-factor type. In this chapter two such preliminary patterns are de- 
veloped. These patterns may then be transformed to any desired form of 
solution. In particular, the rotation to the multiple-factor form is presented 
in Chapter X. An illustration of the rotation from a preliminary pattern 
to the principal-factor form is also given in Chapter IX. 

The two preliminary solutions are the centroid and averoid patterns. The 
theoretical development of the first of these is presented in 8.2, and a list 
of steps for routine calculation is given in Appendix E. Three illustrations 
of the centroid pattern are presented in 8.3. The averoid method of anal- 
ysis, discussed in 8.4, is similar to the centroid method, and, therefore, no 
additional outline for calculation is required. 


8.2. Centroid Method 


The fundamental formula of the centroid solution was first employed by 
Burt* in obtaining a single general factor of the Spearman form. Later, 
when the analysis of psychological tests shifted from one to several common 
factors, Thurstonef provided the complete form for the centroid method, 
which is followed in the main in the present development for the analysis 
of any set of statistical variables. : 

'The name of the methód connotes its close relationship to the mechani- 


* Cyril Burt, The Distribution and Relations of Educational Abilities, 1917, p. 53. 


1 L. L. Thurstone, “Multiple Factor Analysis,” Psychological Review, 1931, pp- 406- 
27; The Vectors of Mind, 1935, Chap. III. ^ 
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- uncorrelated. Then, making the same a 
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cal concept of a centroid, or center of gravity. For this reason, the centroid 
form of analysis can best be described in geometric terms. According to 
Chapter III, the variables may be considered as represented by a set of n 
vectors which are contained in a space of m dimensions, where m is the 
number of common factors; and the scalar product of any pair of vectors is 
the correlation between them, as given in (3.51). The variables may also 
be considered as represented by the m coordinates of the end points of these 
vectors with respect to m mutually orthogonal arbitrary reference axes. 
Since the configuration of the vectors representing the variables completely 
determines the intercorrelations, the reference system may be rotated with- 
out any effect on them. The arbitrary coordinate system may then be ro- 
tated so that the centroid point of the set of n points, along with the origin, 
determines the first axis of reference. Thus it is possible to obtain the pro- 
jection of each of the vectors, or the coordinate of each variable, on the first 
axis of reference through the centroid. 7 : 
Starting with a factor pattern of the usual form (2.16), the correlations 


are reproduced by means of equation (2.22) when the common factors are 
ssumption as in the preceding meth- 


ods of analysis, namely, that the residuals vanish, the observed correlations 
may be written 
(8.1) Tjj = Tjk = üjün + ajsü ko Te. F Rink 

(j,k = 1; Qj opt); 


Where m is the number of common factors. The numerical values of a; 
(t= 1, 2,..., m) are determined by the position of the orthogonal ref- 


erence axes, since aj: is the tth coordinate of variable z;.* Now let the frame 
of reference be so selected that the first axis F; passes through the centroid 


of the system of n points: 


Pi(au, Qi. ++ dim), P2(a21, 422) +++» am) +49 


Palan An?» nm) D 


Any one of the m coordinates of the centroid (say, the E one) +4 n 
age of the corresponding coordinates (the first ones) of the n points. thu 


in general, the ith coordinate of the centroid is given by 


eng ^ g* tS au tebie Da 
n 
k b p 
, i / since it is represented in 
th * More precisely, this variable should be designated by 2j 8 
‘© common-factor space. See 3.8. $ 
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where the summation is from 1 to n on the indicated index when the limits 
are not given specifically. In the present case the centroid lies in the first 
axis of reference, so that its coordinates are all zero except the first, that is, 


(8.3) Dae = Shans = Pie = Daim = 0 
k k k 
The m values (8.2) then reduce to 


1 
PO T an Oy 
k 
there being (m — 1) zeros. Since the centroid lies in the first axis, the first 
coordinate is also the distance of the centroid from the origin. 1 
It is now possible to determine the coefficients of the first centroid fac- 


tor, i.e., the coordinates aj, in terms of the observed correlations. For, sum- 


ming (8.1) for all variables k in a fixed column j of the correlation matrix, 
there results 


Er- en( Sau) + an( Zeu) Pert an| Sem) , 


which, on applying (8.3), becomes 


(8.4) P307 - (Xs) s 
k n 


Then the sum of all the entries in the correlation matrix is simply 


2 
(8.5) »» Dr = (Ze Den) = (X x j 
gu 
Now substituting +, | > mr for 2, amin (8.4), and arbitrarily taking 


the positive sign, the PEATA for aj - be written explicitly as follows: 
bx 
k S; . 
i = : m" G= 1 Bho. ny Mis 
een”? 
a k 


(8.6) 
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AN is the sum of all the correlations in column j of the correlation 
P einn dis the total of all the correlations in the matrix, including the 
bie m in both of these sums. Of course, if the negative sign of the 
a a ie been chosen, the coefficients of the factor would all be changed 

n sign, yielding an equally acceptable factor. Formula (8.6) gives the coeffi- 
cient of the first centroid factor Fı for each variable zj, or the first coordinate 


for each point representing a variable. 

The next step is to get the first-factor 
coordinates are found. Since the residual correlations with one, two, . . . 
(m — 1) factors removed are employed in successive stages of the centroid 
method, the notation introduced for the principal-factor method in 7.3 will 
be employed again. The first-factor residuals are written 


residuals, from which the second 


8. : 
(8.7) Wyk = Tj — Anda = Ajzak + apar T + Ajm@im - 


T : E 
ba residual correlations may be regarded as t 
residual vectors in a space of (m — 1) dimensions—the dimension of the 


pier space being equal to the number of terms in the right-hand mem- 
er of (8.7), or the rank of the matrix of residual correlations, according to 


heorem 3.5. 
h In the (m — 1) residual space, the n po 
ave the following sets of coordinates: 


he scalar products of pairs 


ints representing the variables 


Pildi; dig, -= s 0i) 1 Po(@oe, 023 ++) [PED 


P.(as2, Gnd) +++? anm) * 


Then the (m — 1) coordinates of the centroid of these ” points are 


pia (t= 2,3,..-,m), 


centroid is at the origin in 


). Thus the 
e used directly for calculat- 


z7 
ke all vanish according to (8-3 4 
i m — 1) space, and formula (8.6) cannot c 
‘ng the values of "A second-factor coefficients. It will be noted that in ob- 
taining (8.6) there was involved the division by Saw or n times the dis- 
assumed that the cen- 


D 
was tacitly 
not have been 


ta "C 
eh of the centroid from the origin. It ier a 
Was no ied therwise this ivision 

Possible. t at the origin, for othe 
The immediate problem, then, is to remove tl 


the (m — 1) space, so that the preceding met 


ne centroid from the origin 


3 hod can again be applied. 
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By means of rotations of certain of the vectors about the origin through 
180?— also called reflections in the origin—the centroid can be removed from 
the origin. If the coordinates of a point P;, representing a variable z;, are 


(a, aj - - + Qim) » 
then the reflected point — P;, with coordinates 
(—-aà, —aim..., —ajs), 


represents the variable —2;. Such a variable may be interpreted as the 
original variable measured in the opposite direction. 

Now it is evident from (8.1) that to reverse the signs of the coordinates 
of P; has the effect of reversing the signs of all the correlations of variable z; 
with the remaining ones. Thus the reflection of a variable in the origin is 
accomplished merely by changing the signs of the correlations of this vari- 
able in the correlation matrix. Of course, the same argument holds for the 
residual (m — 1) space as for the original common-factor space of m dimen- 
sions. Hence the reflection of a variable in the residual space is accomplished 
by changing the signs of the residual correlations (8.7) for that variable. 

In an attempt to determine which variables to reflect, Thurstone* sug- 
gests that “it is desirable to account for as much as possible of the residual 
variance by each successive factor.” It should be observed that the same 
idea is involved in the principal-factor method, where the maximization is 
accomplished by a mathematically rigorous procedure.t For the present 
case the maximization principle indicates that the second reference axis 
should pass through a cluster of residual vectors. If there is a clustering of à 
set of residual vectors (i.e., a group of variables having high positive residual 
correlations), which is balanced by a scattering of vectors on the opposite 
side of the origin, since the centroid is at the origin, then the second refer- 
ence axis should be made to pass through this cluster. Thus it would seem 
that the vectors which scatter, opposite to a cluster, should be reflected so 
as to fall in with the group, and then the centroid of the n points will be in 
this group, and the second axis can pass through it and satisfy the foregoing 
principle of contributing to the variances as great a total as possible. The 
second centroid coordinates can then be computed as in the first case. In 
application, those variables which have the greatest number of negative 
correlations would be reflected first, bringing them into the hemisphere of 


* The Vectors of Mind, p. 96. 


t A further analogy between the centroid and principal-factor methods may be noted. 
Each centroid factor is actually the first approximation, in the iterative scheme of 7.4, to 
the principal factor. 
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the cluster. For practical problems, Thurstone suggests reversing “the signs 
of one trait ata time until the number of negative coefficients in the residual 
table is less than n/2,"* that is, less than 2/2 negative signs for any one 
variable, not the entire table. One need not stop with this, however, for 
if it is desired to further maximize" the variance removed by each Brea 
cessive factor, the reflections of variables may be continued until the sum 
A residual correlations for each variable is as large (positively) as pos- 
sible. 

For the remainder of the analytical work it will be convenient to use a 
symbol to designate whether a point, representing a variable, has been re- 
flected in the origin. Let e; stand for the algebraic sign of P;, that is, the 
point ejP; is +P; or —P,. If P; has not been reflected, e; is plus, but, if 
P; has been reflected, then e; is minus. Furthermore, e; may be considered 
as an algebraic operator defined as follows: 


ELS! if z; has not been reflected, 
S — 1 if z; has been reflected. 


"Then e; can be attached to the coordinates of P;, and it can be treated as 
any other algebraic quantity. Thus, if the first-factor residual correlations 
after reflection of certain variables are designated by Tijk (in distinction to 
1';x before reflection), then they may be written as follows: 


(8.8) Tijk = ejex (joo + asks +... Giulio) - 


This result is immediate from (8.7) where each a; and aj, was replaced by 
and then e; and e, were factored out algebraical- 


d, or if both variables were reflected, 
other of z; and z+ was reflected, then 


6a; and erare respectively, 
ly. If neither 2; nor z, was reflecte! 
then 7,;, = rj; but, if only one or the 
"ijk = —;jrg. In other words, Tix = ejexGrix)- 

The (m — 1) coordinates of the centroid of the points, after reflection 


9f variables, are given by 
XS (t = 2,3,...,m). 


rotated about the first axis F1 so that 


Now the system of reference can be 
his centroid.[ Let it be assumed that 


the second axis F; passes through t. 


* The Vectors of Mind, p. 97. 


k t The residual (m — 1) space is orthogonal to the 
ive F», i.e., the first one in the (m — 1) subspace, 
he residual space, and it will be at right angles to Pi. 


first axis of reference F;. The second 
may then be rotated to any position 
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this has been done, there being no need to change the notation for the co- 
ordinates. Then the coordinates of the centroid are 


f 
22,5 0,0. ..,0, 


since the centroid lies in the axis F. From the values for the last (à — 2) 
coordinates the following useful expressions may be written: 


(8.9) X san = Mean LI Y eran =0. 
k k k 


Now the projections of the vectors on the second centroid axis can be 
expressed in terms of the residual correlations. For, summing (8.8) for all 
variables k in a fixed column j of the residual correlation matrix (after re- 
flection of variables) and applying (8.9), there results 


(8.10) Mna = jaja X erare . 
k k 
Then, summing for all columns, 
2 
(8.11) x Mna - dain > aan = (Sac) . 
j k j k k 


From (8.10) and (8.11) it follows that 


or, multiplying both sides by ej, 


(8.12) VT 9 9159 ssa Dy 


where Sj is the sum of all the entries in column j of the matrix of first-factor 
residual correlations and T^ is the total of all the correlations in this matrix, 
the signs of all the entries being those after reflection. The e; indicates that, 
if the variable z; was reflected, then the algebraic sign must be changed, but, 
if the variable was not reflected, then €; is merely +1. In other werde, 
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Sir is the sum of all residual correlations for the unreflected variable z; 
with all other variables. Hence, by defining 


(8.13) ;S; = 6S5, 
formula (8.12) may be put in the form 


S; à 
(8.14) an = T ($51,224: 
1 


In this formula the numerator refers to the sum of the residual correlations 
for the unreflected variable j, while in the denominator T^ still stands for 
the sum of all residual correlations after the sign changes. Formula (8.14) 
gives the coefficients of the second centroid factor Fe for each variable. 
The remaining factor weights can be obtained in a similar manner. In 
general, the residual correlation of rjx after p factors have been removed is 


given by 


p 
(8.15) m E Tjk— » jee = ipid pia H+ F inem - 
t=1 


The centroid of the n points in this residual (m — p) space usually will be 


near the origin. Certain variables may then be reflected in the origin, begin- 
ning with the one which has the largest number of negative residual corre- 
lations. Thus, employing the same principle as before, the centroid may be 
removed from the new origin, and the factor Fp+ı determined so as to con- 
tribute to the residual variances of the variables as great a total as possible. 
After reflection of variables, the residual correlations become 


(8.16) Tpjk = eji oris) = Epik, 


T 2; nor Zi was reflected, or if both were 
f only one or the other of the two vari- 
the projections of the vec- 
fficients of Fp}, can be de- 


where the plus sign holds if neithe 
reflected, and the minus sign holds i 
ables was reflected. Then, proceeding as before, 
tors on the (p + 1)st centroid axis, i.e., the coe 
termined. They are given by 


(8.17) — €Sip _ Si ta 1,2,...,n 
isa = Tn VT, n= Rise), mE 


all the entries in column j of the matrix of residual 


where Sj, is the sum of 
oved, and T, is the sum of all the correla- 


correlations with p factors rem 
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tions in this matrix, the signs of the entries being those after reflection of 
variables. The definition (8.13) is extended to the general case, that is, 


(8.18) pS; = Sip = Sip, 


where the column sum S; is not changed in sign if z; was not reflected, and 
changed if it was reflected. Formula (8.17) gives the coefficients of any 
centroid factor Fp+ for each variable. 

It is evident that formulas (8.6) and (8.12) are special cases of (8.17) for 
p = 0 and p = 1, respectively. The same type of formula is used to cal- 
culate the coefficients of the successive centroid factors. Since a basic prin- 
ciple of the centroid method is to account for as much as possible of the 
variances of the variables by each factor, the variables are reflected in the 
residual subspaces to bring them into a cluster. When the sign changes have 
been made, the centroid of the system of points in the residual space lies 
somewhere in the cluster of variables, and the next reference axis is selected 
through this centroid. Each successive centroid axis is at right angles to 
every one of the preceding axes because the residual space is orthogonal to 
the space of the centroid axes already established, Upon extracting each 
centroid factor, the residual correlations are reduced in magnitude, and the 
rank of each residual matrix is reduced by one. The foregoing development 
was made without any restrictions on the diagonal elements. The number 
of factors ultimately obtained is dependent upon these diagonal values, 
leading to the question of “when to stop factoring.” 

Thurstone* recommends that the largest correlation in each column of 
the correlation 
In each subsequent residual matrix the calculated diagonal term is not re- 
tained but replaced by the largest residual correlation, regardless of sign, in 


each column. This procedure does not furnish a standard for determining 
the number of common factors, 


If, instead of modifyin 
is applied directly, keepi 
ber of factors is determ 
completely resolved. O. 
lation matrix contains 


applied, the resulting solution terminates i 
factors) This feature i 


* The Vectors of Mind, Appen. I. 


matrix be selected for the diagonal element of that column. |, 
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8.8. Illustrations of Centroid Method 


1. THIRTEEN PSYCHOLOGICAL TESTS 

The first illustration of the centroid method is based upon the reduced 
set of thirteen psychological tests of 6.11. The intercorrelations of these 
tests are given in Table 2.2 and are repeated in Appendix E, where the 


TABLE 8.1 
CENTROID PATTERN FOR THIRTEEN PSYCHOLOGICAL TESTS 
Common-Factor O CHUTE 
COEFFICIENTS 
TrsT* 
; i] D a) (2) 
|] «6 | * C3 | Original | Calculated mea 
1l... | 607 —.060 —.443 | .558 .568 —.010 
2. 355 ‘038 | —.266 .203 .198 .005 
3.. 418 .148 —.429 .362 .381 —.019 
don mca ser! ATS ‘083 | —.287 -314 ‘318 | —.004 
Sass | .720 1957 1944 .646 (657 | —.011 
| NM LA 1354 .167 .641 ‘653 | —.012 
T. 721 367 1257 750 ‘721 
8. 705 197 .062 571 .540 
9. 698 409 .252 758 718 
10.. | 455 | —.482 :399 554 599 
T] sei one icu m .537 —.390 .145 449 461 
12., à “487 | —.553 .033 531 544 
Wee sis cil] BA —.368 | —.135 599 608 
Total... sess M m ECT | 6.936 | 6.966 | —.030 
Contribution of] ai | 
factor....... | 4.620 | 1.392 954 luseeees rent tmm e the st 
Per cent of total | | 
original com-| 
munality....| 66.6 | 20.1 1868. eresi 100.4 —0.4 
- f | 


* The names and brief descriptions of the tests appear in Appen. B.1. 


is exhibited. According to the dis- 
it is desirable that "complete esti- 
stimates, the bi-factor commu- 


complete centroid analysis of these data 
cussion of choice of communality in 7.2, 
mates" be employed. Asthe best available e 


nalities of Table 6.12 are used in the centroid analysis. 

The centroid pattern for these thirteen tests is presented in Table 8.1, 
where the centroid factors are denoted by C’s and the unique factors have 
been omitted, Inasmuch as this solution is regarded as a preliminary one, 
the three common factors will not be named. Nevertheless, the common- 
factor space is definitely determined by this solution. The adequacy of this 
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solution is indicated both by the magnitude of the final residuals, given in 
Table E.10, and by the degree to which the original communalities are 
analyzed. The comparison of the calculated and original communalities for 
each test, which is given in the last three columns of Table 8.1, shows that 
the factorization is quite adequate. The total calculated communality ac- 
counts for 100.4 per cent of the original communality, indicating a slight 
overfactorization. The stage to which the factorization should be carried is 
also evident in the actual process of calculation, for, after cach factor is ob- 
tained, the cumulative contributions of the factors can be compared with 
the total original communality. Thus, in the example, the first factor ac- 
counts for 4.620, or 66.6 per cent, of the communalit y, the first two factors 
account for 6.012, or 86.7 per cent, of the communality, while the three com- 
mon factors account for 6.966, or 100.4 per cent, of the total original com- 
munality. By all standards, it is clear that three common factors are suffi- 
cient to describe the given data. 

The form of this solution is very similar to that of the principal-factor 
pattern, including general and bipolar factors. The.centroid pattern might 
thus be retained as a final preferred type of solution. As indicated in the 
preceding section, however, the maximization of the contributions of the 
factors is mathematically more rigorous in the case of the principal solution, 
so it would generally be preferred as a final form. It will be shown in the 
next chapter that this more desirable form can be simply obtained by a 
transformation from some preliminary solution, such as the centroid. 


2. TWENTY-FOUR PSYCHOLOGICAL TESTS 
The centroid pattern for the twenty- 
given in Table 8.2. This pattern may be 
the standards employed in the preceding s 
will also be obtained in Chapter X by ro 
This example gives a practical illustr: 
gence" to a relatively small number 
estimates of the communalities are em 
tion of Table 8.2 is statistically adequa j 
fourth-factor residuals and also by th 
over 97 per cent of the total original c 
of this solution still further, the coeffici 


lated and are presented in Table 8.3. It will be observed that except for 


four psychological tests of 2.8 is 
shown to fit the original data by 
ection. A multiple-factor solution 
tation of this centroid pattern. 

ation of the nature of the ‘“‘conver- 
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TABLE 8.2 
CENTROID PATTERN FOR TWENTY-FOUR PSYCHOLOGICAL TESTS 
* 
Common-Factor COEFFICIENTS COMMUNALITY 
Test® 
: (1) (2) 
9 as ge C | Original |Catculatea| 027? 
—.116 .300 | —.250 .981 .536 045 
—.119 .207 | —.135 .209 .214 | —.005 
—.220 .262 | —.155 .390 .323 067 
-- AI 206 | —.184 .315 .948 | —.033 
—.806 | —.344 .108 .668 .670 | —.002 
—.937 | —.258 .216 .643 .664 | —.021 
—.396 | —.384 124 .786 .745 .041 
—.225 | —.153 | —.060 .930 .516 014 
—.894 | —.240 308 .708 -749 | —.041 
.455 | —.365 | —.136 .503 .572 | —.069 
.397 | —.208 | —.063 .500 .529 | —.029 
.360 | —.149 | —.388 575 537 .038 
“130 | —.099 | —.402 .524 .558 | —.034 
.199 | —.013 .203 .448 321 .127 
.170 .146 .206 .350 .287 .063 
.077 .300 .076 .371 .875 | —.004 
.317 .082 .338 .551 .464 .087 
.307 .248 .072 .484 .460 .024 
125 .129 miu .273 .249 .024 
—.174 .128 .004 .415 .421 | —.006 
114 .080 | —.171 .416 .410 .006 
—.144 .145 136 .415 .430 | —.015 
—.164 129 | —.116 .939 .534 .005 
“151 | —.150 | —.003 .507 .478 .034 
ere ee 11.701 | 11.385 316 
1.600 1.145 1.001 l2: unn temm o reme 
Per .cent of total 
original commu- 
DB TY. ise ees 65.8 | 13.7 9.8 E 97.3 2.7 


* The names and brief descriptions of the tests appear in Appen. Bl. 


TABLE 8.3 
FIFTH CENTROID FACTOR WEIGHTS FOR TWENTY-FOUR 
PSYCHOLOGICAL TESTS 
Test Test g Test 

j ajs ; ajs j 
5 116 
che —.048 
4 —.204 
5. .074 
B.. .001 
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3. EIGHT PHYSICAL VARIABLES 
An additional illustration of a centroid solution is given in Table 8.4 for 
the physical variables of 7.5. This example is included to furnish a simple 
ilustration of a transformation from a centroid to a principal solution, 
which is given in the next chapter. Since the direct principal-faetor solution 
is already available in Table 7.4, a comparison of these two methods of ob- 
taining such a solution can then be made. 


TABLE 8.4 
CENTROID PATTERN FOR EIGHT PHYSICAL VARIABLES 
Common-Factor 
COEFFICIENTS Cousa 
VARIABLE 
a) (2) 2 
i Gi Original | Calculated | 72 
T. —.396 .854 | .846 .008 
2. — .469 .897 |  .889 .008 
3. —.470 .833 | .825 .008 
: —.401 -783 |  .798 —.015 
« .900 | -870 | .868 .002 
3 458 -687 |  .661 .026 
5 444 -521 550 —.029 
^ 333 .979 | 529 .050 
T— 6.024 | 5.966 058 
Contribution of factor....| 4.439 | o TM | EEE ps Bee v 
Per cent of _total original 
communality.....,...: 73.7 25.3 Lio fad 99.0 1.0 


if the average correlation for each variable is 


1 ts communality in formula (8.6), a simplifica- 
tion of the method of analysis results. The sum S;in this formula, Viene the 
ed for rj;, is given by 


ea 1 
(8.19) S - (tL) seu uj k=1,2,...,n), 
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or, denoting the sum on the right by Sj, 


(8.20) S; = (4) Si (f 31,2... Vm). 
The total T of formula (8.6), with average correlations in the diagonal, 
reduces to 

f= 36% 7 = 1, 2.005) 


(8.21) n— l1 


“(eh)” 


Under the foregoing assumption, the first-factor coefficients are given by 


( n )s 
= Ly $ n 
(8.22) a = T = Sip lear 
v( i) 
m= L 
(j= 1,2,...,%), 


om thé general value given 


-(4 ) zai jr, Albanon 


where the prime is used on aj: to distinguish it fr 


by formula (8.6). : 

The simplification of (8.22) over (8.6) is due to the fact that diagonal 
values are not explicitly involved. The values obtained by use of (8.22) are 
identical, of course, with the values which would be obtained by means of 


(8.6) with average correlations in the diagonal. The expression (8.22) can 
also be shown to be equivalent to the ratio of the average of the (n — 1) 
Correlations in S/ to the square root of the average of the n(n — 1) corre- 
lations in 7", Inasmuch as averages are employed in (8.22), the analysis 
based upon this formula is called the averoid method. 

After the coefficients of the first averoid factor have been obtained, the 
first-factor residuals are calculated. Certain variables are then reflected, 
as in the usual centroid method, in order to increase the contribution of the 
Second factor to the residual variance. Then a formula of the form (8.22) 
is applied to the residual correlations for the calculation of the second-factor 
Coefficients. This process is continued until the final residuals are considered 
as insignificant. In contrast to the centroid procedure discussed in 8.2, at 
each stage of the averoid analysis the calculated communalities are re- 
Placed, in effect, by the new average residual correlations. 
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Inasmuch as precise standards for "when to stop factoring" are not 
known for such solutions, some crude basis must be employed. One such 
basis might be the numerical magnitude of the factor coefficients at a given 
stage of the analysis. If the largest of these values are considered to have 


little significance, then the last factor obtained would be rejected as negli- 
gible. 


8.5. Illustrations of Averoid Method 


To illustrate the averoid method, the thirteen psychological tests are 
again employed. It has been seen that very good estimates of the commu- 
nalities are available for this set of variables, and a centroid solution de- 
pendent on such estimates is given in Table 8.1. The centroid pattern based 
on the complete estimates of the communalities will therefore serve as a 
check on the adequacy of the averoid solution. 


TABLE 8.5 


CALCULATION OF THE A, COEFFICIENTS FROM THE INTERCORRELATIONS 
OF THIRTEEN PSYCHOLOGICAL TEsTS 


Vari- 


4.084 2. 806/3.304/4.873 4. 709 4.7 | | 
je | 4. -705 4.769 4.527 2.889'3.615 3. 159 4.504 
@n..| 592) .364  .411| .484| .715| .691 -690, .699| :664| “Bal pa) desl .660 


in the table. The total T" of 


all the Sj. Then the radical in formula (8.22) becomes 


[E~ 
12(50.378) ~ 14664, 


«€— 
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and, upon multiplying this number by each sum Sj, the corresponding 
coefficient aj; of the first averoid factor A; is obtained. 

The first-factor residuals are calculated by the same procedure as outlined 
for the centroid method in Appendix E and are presented in the lower half 
of Table 8.6. It will be noted that more than half of the signs in this part 
of the table are negative, and hence a factor determined from these 
residuals would be inappreciable. In order to increase the contribution of 
the second averoid factor, certain tests are reflected in the origin. The num- 
ber of negative residuals for any test is reduced to six or less by the methods 


TABLE 8.6 
First-FACTOR RESIDUALS AND THE CALCULATION OF THE A: COEFFICIENTS 


Welzcrlslasls|se]|ev[-v]|eJ92 Jy uu -B 


| 


COMNMUBOE 


for reflection outlined in Steps 11-26 of Appendix E. The residuals after 
such reflections are recorded in the upper half of Table 8.6. Then the sec- 
ond-factor coefficients are calculated by means of the formula 


4 n 
(8.23) ais = SiN GH DTI 


t-factor residuals for variable j, and Tj is 
ns being those after reflection. The sym- 
has been reflected then the algebraic 
The sums S/, and the factor coeffi- 


where S/, is the sum of all the firs 
the total of all the residuals, the sig 
bol e; again indicates that if variable 2; 
Sign of the factor coefficient is changed. 


cients aj, are also given in Table 8.6. : . 
The second-factor residuals, before and after reflection, are presented in 


Table 8.7. Then, applying a formula like (8.23), the coefficients ajs of the 
third averoid e A are obtained. The residuals with the third factor 
removed are given in Table 8.8. Since the number of negative residuals for 
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any variable is less than six, no reflections were made in caleulating the 
coefficients a/,. ; : 

In contrast to the magnitude of the third-factor coefficients, all but one 
of the aj,'s are very small. The largest of the latter values is only aj, = 


TABLE 8.7 
SECOND-FACTOR RESIDUALS AND THE CALCULATION OF THE A; COEFFICIENTS 


MESE IE IP oe 6 7 8 9 w | n | 12 | -13 
able 
4 — 043! 100 
“103 016, .005 
- — 010| .002 
- 028! 033 
= 022, 032 
= - — 033, 0 
= j 049) — .020 
= E bia] ona 
B j= 01 i 
= | 170. 025 
=; = 1023| — 042 
: |- 016 
OLB) sna ae 
| 
sh 1 227) .227 
H E | 078 — .078 
E I I I 


TABLE 8.8 
THIRD-FACTOR RESIDUALS 
(Final Residuals) 


Vari- NE 

able} 2 | 2 | 8 | 4 | 5 | 6 | 7 

Locas] eese e ec) as Nm 

2 eme (asini 

3 0|. > | 

4 — 013/2001 :::.:1 | 

5: O91| .027,— 063, j greci: ioni 

8: 1027| .018| 022|- 023 wont ds d einn 

T. 3|—:031| .002, :050 —.009| 038 E r 

Bs — 078| .103| .062| .000'— 048  /040|.. ! b 

98. 013.— 034| .047, 070 ʻoe) /012.— O36. 
10 014|—.045, .032| .003,— .020|— .012| 939, — 370! 
11 .013,—.008.—.068/ .014| .091|— 048l— 006 030 ^0i2 ; 
12 004| .039|— 007| .000.—:049| 027) 047 — 008 138. 
13 —.015| 1063 012-010, .003| -042| .Q16 n07] 007 

4 | j i i 
Sia... | 321) .100| .255| .150| .097| 132| 128 088 3130 133 | .118| 247 | .275 
aty.....| .220| .075) 180 am 068, 003, .090, .002, .002, opa | .083| 174 | .194 
i | | 


.226. It is then clear that the fourth factor may be rejected as negligible. 
This, of course, is also evident from the magnitude of the third-factor residu- 
als, all of which appear to be insignificant. The first three factors are then 
retained in the final averoid pattern, which is exhibited in Table 8.9. 
Further tests of the adequacy of this solution may also be considered. 
The reliabilities of these psychological tests may be compared with the cal- 
culated communalities in order to check whether the factorization is in har- 
mony with the assumptions of 2.3. This check is afforded by the index of 
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factorization (2.14), which is given for each test in Table 8.9. It will be 
noted that only one index of factorization, namely, Hs = 100.3, slightly 
exceeds 100 per cent. Hence this standard indicates that the factorization 
has not been carried too far. The preceding test, however, showed that three 
factors were quite sufficient. Taken together these standards imply that 
three factors, and no more, are required to give an adequate description of 


the original data. 


TABLE 8.9 
AVEROID PATTERN FOR THIRTEEN PSYCHOLOGICAL TESTS 
| CALCULATED INDEX OF 
Common Factors Cossu- ee FACTORI- 
NALITY BASE ZATION 
Trsr 
Ai 
ET Aa Aa Aj tj Hj =100 > 
j Tis 
592 .006 —.371 .488 756 64.6 
.364 .016 —.250 .195 568 34.3 
.411 .115 —.375 .323 544 59.4 
484 .068 — .265 .309 .922 33.5 
.715 .264 .278 .658 .808 81.4 
.691 .367 .203 .653 .651 100.3 
.690 .958 .277 .681 754 90.3 
.699 .205 .086 .538 .680 79.1 
.664 .395 .266 .668 .870 76.8 
.424 —.419 .416 .528 .952 55.5 
.530 — 393 .198 475 .712 66.7 
.463 —.523 .078 .494 .937 52.7 
.660 —.364 —.078 | .574 .889 64.6 
| ae Ca 
Contribution of 
Betor. sava ca 4.391 1.285 .908 iui oe sores axe meme |a idi ns me 


The averoid solution of Table 8.9 may now be compared with the centroid 
Solution of Table 8.1. The corresponding coefficients in these two patterns 
are very much alike, and the calculated communalities of the respective 
variables are not appreciably different. The total contribution of the three 
centroid factors is just 0.351 greater than the contribution of the three aver- 
oid factors, Thus the averoid solution accounts for all but 5 per cent of the 
total centroid communality. These discrepancies are due to employing aver- 
age correlations (in effect) at successive stages of analysis in place of com- 
plete estimates of the communalities. With larger sets of variables, it is to 


be expected that these discrepancies will be even smaller. 

Evidently, one would prefer the centroid solution based upon complete 
estimates of the communalities if such a solution was feasible. The real 
need for a solution such as the averoid becomes apparent when an analysis 
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is attempted of a set of variables for which good estimates of communalities 
are difficult to determine.* 


Another illustration of the averoid solution is given in Table 8.10 for the 
complete set of twenty-four psychological tests. The agreement of this solu- 


TABLE 8.10 
AVEROID PATTERN FOR TWENTY-FOUR PSYCHOLOGICAL TESTS 


Carcu- INDEX OF 
Common Factors Danan: es Facroni- 
Comsou- BILITY ZATION 
VARIABLE NALITY 
Di 
j Ar A: A: As Aj rj; tipet 
.596 | — .143 .273 | — .235 505 .756 66.9 
.976 | — .127 .210 | — .190 | .238 .568 41.8 
421 | — .237 .250 | — .055 299 544 55.0 
.479 | — .240 .216 | — .103 | .344 .922 37.3 
-653 | — .285 | — .345 .112 | .639 .808 79.1 
.648 | — .311 | — .256 .255 | .647 .651 99.4 
-626 | — .366 | — .359 .200 | .695 .754 92.1 
.658 | — .239 | — .166 .014 | .518 .680 76.2 
-646 | — .353 | — .296 .812 | .690 .870 79.4 
.449 -401 | — .393 | — .272 | .591 952 62.1 
.562 -335 | — .236 | — .069 | .489 712 68.6 
.406 -242 | — .113 | — .452 | .493 .937 52.6 
.601 -113 | — .092 | — .387 | .532 .889 59.9 
.431 .198 .022 .851| .349 .648 53.8 
.402 $ T .110 .275 | .289 .507 57.0 
.524 .0 .293 .091 | .376 .600 62.7 
.476 -330 .018 .942 | .394 725 54.4 
.539 -310 .226 | — .004 | .438 .610 71.8 
.456 .184 .115 .017 | .239 .569 42.1 
“614 — .140 .197 .005 | .422 .649 65.0 
b .093 -060 | — .207 | ‘41g .784 53.3 
308 ds .121 .142 .070 411 .787 52.2 
it .164 122 | — .025 | |514 .931 55.2 
i -168 | — .172 | — : 0681 485 836 58.1 
Contribution 
of factor...| 7.441 1.395 1.099 1.080 


oid solution is about 97 per cent of the total 


trating closer agreement than in the preceding 
were employed. 


centroid communality, illus- 
case when only thirteen tests 


PART III 


DERIVED SOLUTIONS: ORTHOGONAL AND OBLIQUE 


CHAPTER IX 
DERIVED PRINCIPAL-FACTOR SOLUTION 


9.1. Introduction 

When any direct solution is obtained, a description of the variables is 
given in the common-factor space thereby determined. The resulting pat- 
tern equations are linear expressions in terms of the common factors for 
theoretical variables which approximate the observed ones. Geometrically, 
any such theoretical variable is the orthogonal projection of the correspond- 
ing observed variable into the common-factor space. The coefficients in a 
pattern equation are the coordinates of a point representing a variable in 
this space. Since the pattern (or reproduced) communalities and correla- 
tions are describable in terms of these coordinates (see 3.8), such values are 
generally different from those used in the original analysis. These discrep- 
ancies have been illustrated in the preceding analyses by the residual cor- 
relations. 

The problem of obtaining à derived principal-factor solution involves the 
transformation of some simply calculated initial pattern. The initial solu- 
tions which will be employed in this chapter are the centroid, bi-factor, and 
averoid forms. Inasmuch as the principal-factor solution can be obtained 
directly, comparison with the transformed solution can be made in each 
case. It will be convenient to designate the three solutions as follows: 


x of initial solution, 


I = pattern matri 
formed princip& 


P, = pattern matrix of trans 
matrix I, 
pattern ma 


Lfactor solution, based upon the 


P trix of direct principal-factor solution. 


i 


d communalities and correlations based upon 


the values in I are involved, whereas P is obtained directly from the ob- 
served correlation matrix. Since the reproduced correlation matrix general- 
ly differs from the observed correlation matrix, it is to be expected that the 
Solutions P; and P will be somewhat different. If the solutions I and P 
fit the original correlations equally well, then the discrepancies between 
P, and P can be regarded as negligible. "These points will be illustrated with 


numerical examples. 

. After presentation of the t 
Cipal-factor form in 9.2, severà 
detail. As pointed out in Chapter V 


In obtaining P;, reproduce 


heoretical basis for transformation to a prin- 
] illustrations of the method are given in 
II, the chief difficulty of the direct anal- 
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ysis for a principal-factor pattern is the large amount of labor involved 
when the number of variables is fairly large. The present method of trans- 
formation is employed to effect a great reduction in the total labor of com- 
putation. The economy of labor becomes more pronounced as the number 
of variables increases. A method for obtaining the roots of a polynomial, 
which arises in-the process of transformation, is given in Appendix F. 


9.2. Transformation of a Given Pattern to a Principal-Factor Solution 


The order of procedure for obtaining a derived principal-factor solution 
consists of the determination of a given factor pattern which can be simply 
calculated, followed by a rotation of this coordinate system so that the 
principal axes constitute the new orthogonal reference system. The defini- 
tion of the principal axes given in 7.4 may be taken for this purpose, name- 
ly, those lines which remain fixed under the transformation (7.12). The 
principal axes may also be considered as the orthogonal reference axes on 
each of which the sum of the squares of the projections of the vectors repre- 
senting the variables is stationary.* 

Employing the geometric ideas of Chapter III, any factor, or reference 
variable, may be regarded as a unit vector in the common-factor space of m 
dimensions. In this space the computed variables of the initial pattern are 
represented by vectors whose lengths are less than unity. Let the reference 
vectors of the given solution be designated by Fi, Fa, . . . , Fm, and let the 
projections upon these axes of the vectors representing the variables be de- 
noted by aj, dj2,..., Qim (j = 1,2)... ; ^). Denote the principal ref- 


erence vectors by Pi, Pa, ... , Pm and their direction cosines with respect 
to the axes Fi, Fs, ... , Fm by 


Qu, 22, = aiig Amt) , Qus; agiro v Ama); .. 
Pas Nas scc, Aed, 


respectively. The problem is to determine the values of these direction co- 
sines so that the transformation fr 


i om the known solution to a principal- 
factor solution can be accomplished. 


The correlation between any variable z; 
may be considered as the sum of the paire 
rection cosines multiplied by the lengths 
factor space. The direction cosines of the 
Gj/h;, . . . , aj /h;), and the length of thi 


and the first principal factor P: 
d products of their respective di- 
of their vectors in the common- 
vector representing z; are (aj/h}, 
S vector is h;. The length of the 


* For general theorems on maxima 2nd minima see Willi Cal- 
duis, 3880, Clap. VIT e William F. Osgood, Advanced 
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reference vector representing P. is, of course, equal to unity. The desired 
correlation may then be written simply as follows: 


(9.1) rj, = agn + Ga «bam G=1,2,...,0). 


This expression will give the coefficients p; (=rjp,) of the first principal 
factor when the values of the \’s are obtained. Inasmuch as the values 
Pi are the projections of the variable z; on the first principal axis, the sum 
of their squares must be maximized. The expression (9.1) for pj may be 
written in condensed form as follows 


(9.1’) Pa = D aidan G= 1,2,...,7). 
8-1 


Squaring both sides of this expression, and summing over the n variables, 
yields 


(9.2) > ph = E» ( S so) . 


j=l j=1 Vel 


For convenience, the left-hand member of this expression will be desig- 


nated by Ax. ae 
The sum of the squares of the projections, Ai, must be maximized under 


the condition that the sum of the squares of the direction cosines is equal 
to unity. This condition may be written in the form 


(9.3) B= > -1=0. 
s=1 


In maximizing 4; by the method of Lagrange's multipliers,* the function 
(9.4) 2w = À1— ui 


meter which will be determined 
function of the m variables \n, 
ditions for w to be stationary, 


is employed. In this expression, # is a para 
in the following analysis. Consider w as a 
An... , M and write the necessary con 
namely, 


aw _ 1/941 _ 2) = 0 s= 1,2 m). 
(9.5) oe oo ENa ( ‘eas ag , ) 


* Ibid. 
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Now, the derivative of A; with respect to any variable X; is given by 


94i. : V . 


(9.6) 
= 2 >) alanu + Qj2À21 + xd. + jm Nm) , 
j=1 
and the derivative of B, is 
9B; - m). 
(9.7) ha 7 M (5:5 152; 54, m) 
Substituting (9.6) and (9.7) in (9.5), the latter conditions may be written 
explicitly for s = 1, 2,..., m, as follows: 
Au(Za?, — u) + Mabaa; Teo. A mi Zain =0, 
(9.8) AnZajoaj + alla} — i) +... H AmtZAj20jm =0, 


AnXajsaj + AnXajsa;2 SP essc Ami (Fam — u) =0, 
where each of the summations extends from j = 1 to j = n. 
A necessary condition for the m equations (9.8) to have solutions (in 
which not all of the values X are zero) is the vanishing of the determinant 


of coefficients. This condition leads to the characteristic equation, which 
may be written in the form 


Xa — n) Zajaz Lajidin 
(9.9) f(x) = Zanaz (Zaja — u) ...  Xajüjs 26 
Zaima; La jmQj2 ee (Xa, — à) 


Upon expanding:this determinant, this equation can be written as follows: 


(9.10) fü) = u" — eun?! + em =. (pe = 0, 

where the c's are positive numbers.* Each of the m roots of this equation 
can be obtained to any desired degree of accuracy by methods such as those 
outlined in Appendix F. 

Evidently any root p, (s = 1,2,..., m) of (9.10) will make the determi- 
nant in (9.9) vanish and yield a solution for the unknowns in (9.8). The solu- 
tion is labeled (An, Aa, . . . » Amı), however, only in the case that the largest 

* See Appen. F. 
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root pı of (9.10) is employed. When the other roots p2, us). - + » m Of (9.10) 
are employed, the sets of direction cosines Qus, Nes > + + Ama) +++ y Qnm 
Nan, ss sy Amm) Of the remaining principal axes are obtained. This follows 
from the preceding form of analysis by a simple substitution of indices. For 
example, in order to obtain the direction cosines of the second principal 
axis, all that is necessary is to replace piu Mn Ai, Bi by pio No A2, Ba, re- 
spectively, and equations (9.1’) to (9.8) follow as before. The new equa- 
tions (9.8) will be formally identical with the old, except that the unknowns 
will be (Aiz, A22, . . . Ama) instead of (Au, Ne +++» Amı). The determinant 
of coefficients, nevertheless, will be the same and the characteristic equa- 
tion (9.9), or (9.10), will be unchanged. 
It is clear, then, that the problem of making any sum of squares, Ai, Az, 
., Am, stationary leads to the same characteristic equation. Thus the 
direction cosines of the respective principal-factor axes may, in effect, be 
obtained by employing the roots Hs (s21,2..- in) of (9.10) successively 
in (9.8), changing the second subscript of the A's in each case to agree with 
the index of u. 
Furthermore, it can be shown that the roots of the characteristic equa- 
tion are equal to the respective sums of squares, i.e., He = As (s = 1, 2, 
. m). Hence if a root is very small, indicating that the contribution of 
the corresponding factor is insignificant, the direction cosines of that factor 
axis need not be obtained. 
When all the desired Ms have been determined, it is then possible to 
ncipal-factor form. The first column 


transform the given solution to the pri : A ; madti 
of the transformed pattern is given by equation (9.1) in which the direction 
cosines of the first principal axis are now known. The complete transforma- 


tion can be exhibited most conveniently in matrix form, as follows: 


an dip... dim || ET ELLE pu Piz +++ Pim 
dan ds... dam | |] Av Mees om || _ |] Par Pee Pom 
Ani a2... Anm Aml Am2 - + Amm Dui Pnz +++ Pam 
I I I 
(9.11) A z T - P 


given solution, T is the matrix of trans- 


formation, and P is the resulting principal pattern matrix. The multiplica- 
tion is performed by the conventional row-by-column procedure, i.e., the 
element p; is the sum of the paired products of the elements in the jth row 
of A by the corresponding elements in the kth column of T. This completes 
the analysis for obtaining @ derived principal-factor solution. 


The pattern matrix A is that of the 
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9.3. Numerical Rotations to the Principal-Factor Form 

When a principal-factor solution is desired, the most economical pro- 
cedure for obtaining this form is by a rotation of a simply calculated initial 
solution. The reduction in labor over the direct method of Chapter VII is 
especially great for a large number of variables. The initial solutions con- 
sidered for this purpose are the centroid, bi-factor, and averoid patterns. 


1. ROTATION FROM CENTROID SOLUTION 


In order to illustrate the method of transformation very simply, a set of 
only eight variables involving two factors will be employed. These data 
are the physical variables which were analyzed directly into a principal- 
factor solution in 7.5. Thus a comparison between the direct and indirect 
solutions may be made. The centroid pattern of these variables, which is 
given in Table 8.3, is taken as the initial solution for purposes of rotation. 
This pattern may be exhibited as follows: 


C Cs 
830 —.396 
818 —.469 
-TTT —.470 
c= -798 —.401 ; 
-786 .500 
672 458 
594 444 
647 333 


where C is used to denote the cent; 
the centroid factors whose coeffici 
In the process of obtaining the 


roid pattern matrix, and Ci, C; represent 
ents appear in the columns, 


8 


£ 8 
a, = 4.4394 , X anan = —.2176., Ya = 1.5263. 
j21 del z 
The characteristic equation may then be written in the form 
(9.12) Stu) =| 44394 — a) — 2175 


—2175 (1.5263 — al = 95 
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or 
f(u) = pè — 8.9657, + 6.7285 = 0. 


The two roots of this quadratic equation are readily found to be 


m = 4.4556, p= 1.5102. 


Employing the larger root, the direction cosines of the first principal- 
factor axis can be obtained by means of equations (9.8), as follows: 


(9.13) l —.0162An — .2175An = 0, 


— 217533 — 2.929318 = 0. 


knowns An, Ag, it is 


In order to obtain unique solutions for the two un i 
which in this case is 


necessary to employ the auxiliary condition (9.3), 
simply 


019 MEME 


The two homogeneous equations (9.13) have a matrix of coefficients of 
rank one, as is evident from (9.12), and so one of these equations may be 
used to solve for one unknown in terms of the other. This solution will also 
satisfy the second equation.* From the first of equations (9.13), the solution 


(9.15) An = — 13.43 


te number of values Xi corresponding to 


is obtained, which gives an infini 1 1g 
n a determinate solution, the condition 


arbitrary values of àn. To obtai 
(9.14) is now employed, yielding 
(—13.43^2)* +n = 1, 


or 
Aa = +.07426 . 


Substituting this value in (9-15) gives 
Me 7.9973, 


where the minus sign is associated with the positive value of ^a and vice 
versa. The positive sign will be taken for the larger direction cosine for 


convenience, i.e. Au = 9979, An = — 0748. i 
The second Ns - 1.5102, is next employed in order to determine the 
direction cosines of the second principal-factor axis. Substitution of u» for p 


* L. E. Dickson, Modern Algebraic Theories, p- 61. 
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in (9.8), and replacing the second subscript on the X's by 2, gives the follow- 
ing pair of simultaneous homogeneous equations: 


2.92902X, — .2175r22 = 0 3 
(9:16) — J2175X + .0161Ag = 0. 


The solution of these equations, under the restriction 


MM, 
is M2 = .0740, Av = .9972. 
The direction cosines having been determined, the matrix of transforma- 
tion can be written as follows: 


To =|| 9973 .0740 
e || —.0743 .9972 || ° 


Then, upon multiplying the centroid pattern matrix C by Tc, the follow- 
ing principal pattern matrix is obtained: 


Bi B 
.857 —.333 
.851 —.407 
810 —.411 
.826 —.341 

Bm J47 — .557 
.636 ^ .506 
.559 487 
621 .380 


It will be noted that there was an ambiguity in choice of algebraic signs 


rection cosines of the principal-factor axes. 
ular set of signs may now be indicated. If the 
of a principal axis are changed, the elements 
in the matrix Po are changed in sign. The 
t changed, however, by the reversal of all the 
a pattern matrix, as pointed out in 2.5. 

ipal-factor pattern, the coefficients of which are given 
in the matrix Po, agrees very closely with the solution obtained directly in 


Chapter VII and exhibited in Table 7.4. The coefficients of the first factor 
in the two cases have a maximum discrepancy of only .002, while the coeffi- 
cients of the second factor have a maximum difference of .009. 


adequacy of the solution is no 
signs in a column of 
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2. ROTATION FROM BI-FACTOR SOLUTION 

An example will be presented now in which a derived principal-factor 
solution is obtained from an initial bi-factor pattern. The illustration is for 
the same set of eight physical variables which were just employed in passing 
from a centroid solution to a principal-pattern. It will be evident then that 
the derived solution from either of these initial patterns is essentially the 
same as the principal-factor solution obtained directly in Chapter VII. 

The bi-factor pattern matrix for the eight physical variables is taken from 
Table 7.2 and may be written as follows: 


Bo Bi B» 
.691 .614 0 
.591 .740 0 
.581 .704 0 
598 .652 0 
B=|\ 694 0 .623 
611 0 .560 
562 0 .453 
.596 0 .473 
For these data the characteristic equation (9.9) takes the form 
(3.0479 — u) 1.6605 1.8110 
(917) f(u) =| 1.6605 (18453 — 2 0 20. 
1.3110 0 (1.1307 — u) 


In the determinant of the characteristic equation arising from a bi-factor 
pattern there will generally be zeros for the elements off the diagonal, except 
for those in the first row and column. This property holds true when the 
Subsets of variables measuring the group factors are distinet. When there is 
Some overlap of group factors; then some small values will appear for these 
elements. 

Upon expansion of the determin 
may be written 
(9.18) flu) = p? — 6.02394? + 6.68104 — 0701 — 0. 


d by the methods of Appendix F, yielding the 


ant in (9.17), the characteristic equation 


This equation may be solve 
three roots 


m = 4.5631 , 
(9.19) p = 1.4502, 
ps = 0106. 
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Then, substituting the largest root for u in equations (9.8), the following 
three equations arise: 


—1.5152Aq + 1.6605A4 + 1.311044; = 0 
(9.20) 1.660534 — 2.71781 


1.311015 — 3.4324» = 0, 


LU 
o 


where the direction cosines of the first principal axis with respect to the 
three bi-factor axes are denoted by Xo, Au, Ag. The solution for Xo, Xii, 
^a is subject to the restriction 


(9.21) Mit M+ A, I. 


The last two of equations (9.20) may be considered as linearly independent, 
and the variables Xi, Aa solved in terms of Nor 


The resulting expressions 
may be written 


L 1.660534 _ 213110 — " 
An = 27178 C 61097 Mo: and `a = "Edd = -38195do - 
By making use of the condition (9.21 


), the direction cosines of the first prin- 
cipal-factor axis are found to be 


Aor = .8113 , An = .4957, and Aa = .3099. 


The direction cosines of the second principal axis are similarly obtained, the 
values being 


^o = .1678 , Aw = —.7053, and Ao = .6888 . 


It will be observed that the third root 


-0106, is very small in comparison with the others. Inasmuch as this value 


is equal to the contribution of the third principal factor, this factor will not 
be obtained because of its insignificance.* 


The two sets of direction cosines are 
transformation: 


of the characteristic equation, us = 


recorded in the following matrix of 


|.8113 1078 

T, = || .4957 —.7053 

| -3099 -6888 
* Precise methods for determining the Significance of 


Harold Hotelling, “Analysis of a Complex of Statistical 
ponents,” Journal of Educational Psycholog; 


principal factors are given by 


Variables into Principal Com- 
y, 1933, pp. 417-42. 
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By multiplying the bi-factor pattern matrix B by Ts, the derived principal- 
factor pattern matrix is obtained, as follows: 


Pi Ps 
865 —.317 
846  —.423 
.820 —.399 
.808 —.360 
Ps =|| 756  .546 
.669 -488 
.596 -406 
.630 — .426 


It may be observed that the principal-factor pattern derived from the bi- 
factor solution agrees very closely with the direct solution given in Table 
7.4. The sum of the contributions of the derived principal factors P; and Pa 
is 6.011, or 99.8 per cent of the total original communality used in obtaining 
the direct solution. The slight discrepancies between the derived and direct 
principal-factor patterns are due to the fact that the total communality of 
the latter solution did not agree exactly with the original communality, 
which the former solution reproduces. 

From the foregoing illustrations, it is apparent that a principal-factor 
form of solution may be derived from cither a centroid pattern based upon 
good estimates of communalities or a bi-factor pattern. If the communali- 
ties of two initial solutions are the same, then the derived solutions will 
agree, Such agreement is illustrated by the two derived matrices Pc and Pz, 
the minor discrepancies being due to the slight variations in the communali- 


ties calculated in the initial solutions. In case no initial solution of the pre- 
Ceding type is available and a principal-factor solution is desired, a pre- 


liminary pattern such as the averoid may be employed. A derived principal- 
factor solution may be considered as appropriate provided it is based upon 
any initial pattern which is statistically adequate. 
3. ROTATION FROM AVEROID SOLUTION 
To illustrate the method of rotation to à principal-factor pattern fora 
fairly large set of variables, the twenty-four psychological tests are em- 
ployed. After obtaining such à derived solution for these variables, it may 
be compared with the first two principal factors which were obtained di- 
Tectly and are given in Table 7.5. Although both a bi-faetor and a centroid 
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solution are available for these data, the present analysis will be based upon 
the averoid solution of Table 8.10. The averoid was selected as the initial 
form of solution in order to indicate the procedure when good estimates of 
the communalities are not available. The total time required to calculate a 
direct principal-factor solution for these data was estimated to be two hun- 
dred hours, while the time required for the computation of the averoid solu- 
tion and the subsequent rotation was about twenty-five hours. The econo- 
my of labor for a set of variables of this size is clearly evident. 

The necessary sums for the characteristic equation (9.9) are calculated 
from the averoid pattern of Table 8.10, producing 


(9.22) f(u) = 
(7.4406 — u)  —.3136 — .2753 .0360 
—.3136 (1.3950 — 4) .0413 —.3222 E 
—.2153 .0413 — (1.0087 — u) —.0917 
.0360 —.8222 —.0917 (1.0799 — 4) 


This determinant can be expanded most simply by Laplace's development 
by columns,* using the first two columns as a base, and then the character- 
istic equation becomes 


(9.23) f(u) = wt — 11.014245 + 30.52604? — 31.7349, + 11.1727 = 0. 


The complete procedure for the transformation from the averoid pattern 
to the principal-factor form is described in Appendixes F and G. In the 
first of these appendixes a general method for calculating the roots of a 
polynomial of any degree is presented. As an illustration of this method, 
the roots of the characteristic equation of the present example are evaluat- 
ed. There are four roots in this case so that four sets of equations of the 
form (9.8) arise. The solution of Such sets of simultaneous equations is pre- 
sented in Appendix G.2, in which the data of this example are used again 
asanillustration. The derived principal-factor pattern is given in Table 9.1. 

Since the axes representing the derived principal factors are obtained by 
rotation of an initial set of axes in a fixed common-factor space, the calcu- 
lated communalities in the two cases are, of course, the same. This agree- 
ment may be seen by comparing the corresponding values of Tables 8.10 
and 9.1. It then follows that the total contributions of these two sets of 
factors must be equal, which is again evident from these tables. As would 


* See, e.g., L. E. Dickson, First Course in the Theory of Equations, p. 122. 
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be expected, however, the contributions of the individual factors are allo- 
cated differently. 

The coefficients of the first two factors of the above pattern may be com- 
pared with the corresponding values of the direct solution given in Table 


TABLE 9.1 
DERIVED PRINCIPAL-FACTOR PATTERN FOR TWENTY-FOUR 
PsYCHOLOGICAL TESTS (BASED UPON THE 
AVEROID PATTERN OF TABLE 8.10) 


Common Factors 
Carcu- 
Test LATED 
yaba | eami | sisi Ek ei 
General | Rigidity | Spatial | Memory | Naurry 
j Pi Pi Ps P. Aj 
lis .588 | — .079 :350 | —.174 | .505 
2.. .371 — .047 .273 | —.153 | .288 
3. .421 .109 .821 | —.083 .299 
4. .480 .087 .298 | —.1382 | .344 
5. .682 .809 | — .252 | —.120 .639 
B... .676 .896 | — .182 .009 .647 
T .661 .427 | — .259 | —.092 .695 
8.. .676 ` 193 | — .079 | —.131 .518 
9.. .676 .457 | — .151 .043 .690 
10... .442 — .449 | — .425 | —.118 .592 
Ilo .553 — .812| — .285 .063 .488 
12... .453 — .454 | — .089 | —.270 | .492 
13. 1594 | — .323 | — .041 | —.271 | .532 
14.. .492 | — .000 | — .071 .407 | .349 
15.. .888 | — .053 .024 .367 | .289 
16.. .506 | — .089 .259 .210 | .375 
Im. 1450 | — .170| — .091 .382 | .394 
18.. .512 — .319 .181 .226 .438 
19.. .443 — .142 .086 .125 .239 
20.. .615 .116 .161 .062 .421 
21... .591 — .230 .082 | —.091 .417 
22.. .608 -.085 .174 .054 .410 
Da... LI .688 .068 1183 | —.049 | .514 
p. epp .648 = .183:} — .179 .007 .485 
Contribution 
of factor..| 7.467 1.591 1.093 BBD, Ys iarctecaresace 


7.5. The agreement of the individual coefficients is remarkably close in the 
Case of both factors, and the total contributions of the respective factors 


are practically the same. It would therefore appear that a very close ap- 
Proximation to the principal-factor solution calculated directly from the 
correlations can be obtained by the preceding rotational method. From the 
foregoing discussion it would appear that, if a principal-factor pattern is 
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desired for a large set of variables, a satisfactory solution can be obtained 
effectively by rotation of an initial averoid pattern. 

Since the first two factors agree with those obtained by the direct anal- 
ysis, they may be named “General” and “Verbal Rigidity,” as in 7.5. The 
third factor has been designated as "Spatial" because the largest positive 
coefficients appear for spatialtests. This factor might also have been named 
“Spatial-Speed” because of the negative coefficients for the speed tests. 
The last factor has been called “Memory” because of the consistent posi- 
tive weights for the six memory tests. 


1 


CHAPTER X 
ORTHOGONAL TRANSFORMATIONS AND THE 
MULTIPLE-FACTOR SOLUTION 


10.1. Introduction 


In the preceding chapt 
principal-factor form were discusse 


er rotations from simple initial patterns to the 
d. Such rotations were expressed com- 
pletely in analytical form because of the exact mathematical properties of 
the principal-factor solution. Other preferred types of solution may not be 
defined in such precise mathematical terms. The transformation of an ini- 
tial pattern to a preferred type such as the multiple-factor solution, there- 
fore, cannot be developed directly as in Chapter 1X but must be constructed 
somewhat subjectively. The guiding principles in formulating such a trans- 
formation are given by the criteria set forth in Table 5.3. 

Before considering the applications of transformations to factor analysis, 
the theoretical development of orthogonal rotations is first presented. In 
10.2 the rotations in the plane are developed and are used as the basis of the 
rotations in three- and higher dimensional spaces in 10.3 and 10.4. Specific 
application to the multiple-factor form of solution is made in 10.5. Other 
types of transformations leading to multiple-factor solutions are available* 
but will not be treated in the present work. Multiple-factor patterns are 
obtained for the eight physical variables, the thirteen psychological tests, 
and the complete set of twenty-four tests. Tn the first and third of these 
illustrations the centroid is used as the initial solution, whereas for the sec- 


ond example the averoid solution is employed. 


10.2. Rotations in a Plane 


When an initial factor patte 
may be represented as points in 
as the coordinates. Then a trans 


rn involves only two factors, the variables 
a plane, with the coefficients of the factors 
formation to some other form of solution 


implies the representation of these points with respect to the axes denoting 
the new factors. Inasmuch as the origin is assumed to be fixed, such a trans- 
formation is merely @ rotation of axes in this common-factor space. The 
transformations considered in this chapter are orthogonal, and hence final 
patterns involve uncorrelated factors. 

* See, e.g., L. L. Thurstone, “A New Rotational Method in Factor Analysis," Psycho- 


metrika, 1938. 
215 
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In deriving an orthogonal transformation in the plane, the required terms 
may be indicated conveniently as follows: 


T^, F} = axes representing the original factors, 
P; = point representing variable z; (j = 1,2,..., n), 

aj, aj» = coordinates of P; with respect to Fi, Fa, i.e., the coefficients in the original 
factor pattern, 

My, Mz = axes representing the final factors, 

bj, bj» = coordinates of P; with respect to Mi, M», i.e., the coefficients in the final 
factor pattern, 

0 = angle of rotation from original to final axes. 


E 


(Q,, Qa) 
pon b, 


M, 
Fia. 10.1 


Employing the above notation, 


any variable may be represented by a point 
Pj, referred to either System o 7 f M 


f reference Fy F; or My, M hown in 
: : Mi, Ma, as shown 
Figure 10.1. The problem 1s to express the coordinates (a, bj2) in terms 
of the original coordinates (aj, aja). This is equivalent to obtaining the final 
factor pattern from the initial one. 

The required transformation is accomplished by making use of the fol- 
lowing property on Projections of lines: the sum of the projections upon 2 


straight line of the segments of any broken line connecting two points is 
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equal to the corresponding sum for any other broken line connecting the 
same two points. The points O and P;are joined by two broken lines, name- 
ly, ORP; and OSP;. It follows that the projections of these broken lines 
along any direction are equal: 


(10.1) Proj. OR + Proj. RP; = Proj. OS + Proj. SP;. 


If the direction is taken, first, as the positive axis of M; and, second, as 
the positive axis of Ms, the resulting expressions are 


(10.2) k cos 0 — RP; sin 0 = 0 + SP;, 


OR sin 0 + RP; cos 0 = OS + 0, 


Now, employing the definitions of the coordinates, 


OR = ay, RP; = àjs, SP; = ba, OS = bis, 
the result may finally be written in the form 


(10.3) | bi = aj cos 6 — aja sin 0, 


bj; = aj sin 0 + aj» cos 6, 


in which the angle 8 is negative. There are n such sets of equations corre- 


Sponding to the number of variables. D. 

In the above analysis it was convenient to take 0 negative in order to get 
direct expressions for the final coordinates in terms of the original ones. If 
this angle 6 is denoted by —¢, then formulas (10.3) become 


l ba =  aancosó + aj sin ó, 


(10.4) ba = —ap sin $ + Gi cos}. 


It may be noted that these formulas reduce to (10.3) if the angle ¢ is nega- 
tive. Hence (10.4) may be taken as the general equations of transformation. 
This transformation may also be expressed in matrix notation, as follows: 


cos ó E 


(10.5) lbi bill = lan ajll * sin $ cos ó 
or 
(10.6) M - AT, 


Where M is the n X 2 matrix of final factor coefficients, Ais then X 2 ma- 
trix of initial factor coefficients, and T is the 2X 2 matrix of the transforma- 
tion. Formulas (10.4) express the new coordinates (bj, bj») as functions of 
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the old coordinates (aj, aj»), involving only one parameter ¢. Thus, when 
a value is assigned to 6, the final factor pattern can be obtained. The man- 
ner in which such an angle of rotation is selected is demonstrated in 10.5. 

To illustrate the analytical properties of orthogonal transformations, the 
foregoing equations may be written in the general form (3.16) as a set of 
linear homogeneous equations: 


ba = Mids + Matje , 
(10-7) ta = Mj + No20; , 


-where the A's are the coefficients expressed in terms of the angle $. These 
coefficients, by rows, are the direction cosines of M, and M; with respect 


to the original axes F, and P; and are conveniently indicated in tabular 
form as follows: 


Fy, F: 


Mı | `n = coso An = sin ọ 
M: | M2 —sinó Aw = cosd 


Inasmuch as the direction cosines satisfy the 


conditions (analogous to 
eq. [3.15]) 


Mi M, = 1, uio + Aes = 0, and atl, 


it is clear that (10.7) represents an orthogonal transformation. 

Besides enabling the analyst to rotate one solution involving two factors 
to another, the above type of transformation forms the basis of the analysis 
for a solution including several factors. It will be shown that a transforma- 


tion in higher dimensional space may be reduced to successive rotations in 
planes. 


10.3. Rotations in Three-Space 


In the case of three factors the transformation from an initial to a final 
pattern may be exhibited as follows: 


ba = Auda + aaj + Nadja, 
(10.8) bja = Meds + Maij; + Aaadjs , 
bjs = Asay + Aaajs + Nast js , 


where the notation of 10.2 has been extended to an additional factor. The 
transformation (10.8) may again be expressed in matrix form 


(10.9) M = AT, 
where, now, M is n X 3, A isn X 3, and T is 3 x 3. 
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The elements of T are direction cosines which must satisfy the six inde- 
pendent conditions 
MEM EM, 
fo + Me + Mz = 1 

Ms + Mat = L, 

Audie H Aada + Asda = 0, 

Audis + Aedes F Aada = 0, 

Adis + Moles + Mes = 0. 


, 


Each of these equations is an instance of the following general expression: 


Nu Na + Asat = Sat (s t= 1, 2,3, s £t); 


which may be written compactly in the form 


3 
(10.10) DS Mudur = ôt (121,23, s £t), 


u-l 


Where à,, is the Kronecker à which is equal to unity if s = t and equal to 
zero if s + t. Thus the nine coefficients of (10.8), being subject to six con- 
ditions, afford only three degrees of freedom of rotation in ordinary space. 
Explicit equations for the b's in terms of the a's, involving only three in- 
dependent parameters can be obtained.* Since such equations are not em- 
ployed in practical analyses, they are not exhibited here. 

A form of transformation that is not only practical but which can be 
readily generalized to any number of factors will now be discussed. The 
fundamental principle underlying this method is that the result of succes- 
sive orthogonal transformations is itself an orthogonal transformation. This 
final transformation is said to be the product of the successive rotations. 
Since a planar rotation is the simplest type, à transformation in three-space 
is built up from rotations of essentially this form. Thus a transformation in 
Ordinary space may involve the displacement of any two axes about the 
third, being, in effect, a rotation in a plane. Finally, a product of such rota- 
tions may be taken as the complete transformation. 

The rotations can be arranged in a systematic order so that each axis is 
Totated with every other axis only once. The three rotations of pairs of axes 
in ordinary space may be indicated conveniently in the following manner: 


Old Axes Angle of Rotation New Axes 
FiF2 biz YiYs 
YiFs 3 MiYs 
YY; O23 M:M; 


* Virgil Snyder and C. H. Sisam, Analytic Geometry of Space, 1914, p. 42. 
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It will be noted that the angle of rotation is denoted by 6 with subscripts 
corresponding to the numbers of the axes involved in the rotation. The 
first rotation is made in the plane of F, and F», leaving F; unaltered. The 
new axes in this plane are designated by Y; and Y». Since F; is perpendicu- 
lar to the plane of F, and Fs, it is perpendicular to any line in this plane. In 
particular, F; is perpendicular to the new axis Yi. The next rotation is 
made in the plane of Y; and Fs, leaving Y» unchanged. The new first axis, 
denoted by M;, may be regarded as final because it is the result of rotations 
with each of the other axes. The last rotation transforms Y; and Ys into 
the final coordinate axes M» and M3. It will be observed that the Y's are 
merely auxiliary axes and, taken alone, do not form an orthogonal system. 
Both sets of axes Yi, Y», F and M;, Y», Y; are orthogonal, and either one 
may be taken, in some instances, as the final reference system. The solution 
ordinarily desired, however, is one based upon the complete transformation 
of the original axes Fi, F», F; to the final Mi, M», M3. 
Denoting the matrix of transformation of F, and F;, leaving P; un- 
changed, by 
cos ĝi —sin 6i. 0 
Ti; = |sin (i. cos. 0 
0 0 1 


D 


the first of the above rotations may be denoted by 


(10.11) Y - ATi, 


where A is the initial pattern matrix and Y is an intermediate matrix of 


coordinates with respect to Yı, Ys, F4. The second and third rotations may 
be designated similarly, as follows: 


(10.12) Z = YT, 
and 
(10.13) M = ZTz, 


where Z is another intermediate matrix, M is the final matrix of coordi- 
nates, and 


cos 3 0 —sin 6 
Tu = 0 1 0 
sin is 0 cos 6, 


1 0 0 
, Tos = |0 cos 0; —sin Oes 
O sin 623 cos O03 


The three preceding rotations ma: 


. e pre y be combined into a single transforma- 
tion. Substituting (10.12) for Z in 


(10.13) gives 
M = YTu4TA, 
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and substituting (10.11) for Y in the last equation yields 
(10.14) M = ATu;TuTs. 

Denoting the product of the three successive rotations by 
(10.15) T = TuTaTss, 


the expression (10.14) reduces to (10.9). In practice this matrix T cannot 
be obtained directly, but the final pattern is calculated by means of the 
successive rotations. When each of the partial transformations has been 
obtained, however, the complete product matrix T may be recorded and 
used as a check on the coefficients of the final factors. 


10.4. Rotations in Higher Dimensional Space 
w be generalized to a common-factor space 


The preceding methods can no n 
t the initial pattern matrix be denoted by 


of m dimensions. In this space le 
A= lall GLB S512.. m) 


and the final matrix by 


M = |lbill G=1,2%..- 0; 5212...,m. 


Then the transformation from the initial to the final pattern may be ex- 
pressed as follows: 


(10.16) ba = Y, Ni 
(12..,m5-7L2...,m) 


or in the equivalent matrix form 


(10.17) M=AT, 


where the matrix of transformation 1s now 


hn 9 aou Aim 
or Age e Am 
T z . eee . . 
Ami Am2 Amm 
are the direction cosines of the final reference 


The set: : lumns. 
ca o ‘ t to the original axes Fy, Fs... Fn. 


axes Mi, Mo, ..., Mm with respec 
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These direction cosines are subject to the following set of independent con- 
ditions for orthogonality of the matrix T (see 3.5): 


pem, 
i 


WA Il 


S 


= 8,1 
(10.18) Do hadu = bu ( 
u-l 


Since s < ¢ and these indices range from 1 to m, the number of such condi- 
tions is >) +m or m(m + 1)/2. There is a total of m? parameters in 


matrix T, and, since these are subject to m(m + 1)/2 restrictions, there 
remain 


m(m +1) _ m(m — 1) 
(10.19) m? — 3 - 3 


degrees of.freedom of rotation in m-space. The number of independent 
parameters given in (10.19) may be associated with the same number: of 
rotations in planes. The planes in which the rotations are made are de- 
termined by all possible pairs of reference axes. These rotations then in- 


volve m angles corresponding to the number of independent parameters. 


Asin the case of three variables, the above rotations may be organized in 


a systematic manner. For example, when four factors are involved, the 
Scheme may be indicated as follows: 


Old Axes Angle of Rotation New Axes 
FiF: i Y;Y, 
YiFs 413 “Ys 
ZW, Ou MY, 
YY; 055 ZZ; 
Z:Y4 [71 MZ, 
ZZ, 03. MM, 


10.5. Applications to the Multiple-Factor Solution 
In obtaining a multiple. 
conformity with the discu. 


lution shall satisfy the c 
factors, 


-factor solution, the reference axes are chosen in 
ssion in 5.6. It is first necessary that the initial so- 
riteria of composition of variables, parsimony of 
and uncorrelated factors. Then an orthogonal rotation of such an 
initial pattern, in its common-factor space, will preserve these properties. 
The purpose of the transformation is to obtain a final pattern which also 
satisfies the criteria of low complexity, level contributions of factors, and 
hyperplanar fit (i.e., as many zeros as possible in the columns). 

Although any pattern with uncorrelated factors could be used as the ini- 
tial solution from which to rotate to a multiple-factor pattern, in the pres- 
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ent treatment only centroid and averoid initial patterns are considered. The 
procedure is begun by plotting the points representing the variables in the 
plane of the first two initial factors F, and Fə. Since in an initial solution 
of the centroid form the second factor has both positive and negative 
weights, the points will lie in the first and fourth quadrants (see Fig. 10.6). 
The first rotation is then made through an angle 61» such that all variables 
will have positive projections on the new axes Yi, Ys Usually the angle 
6:5 so selected will be about —45°. Then, by applying equations of the type 
(10.4), the coordinates with respect to Yı and Y; are obtained. The next 
rotation is made in the plane of Y; and Fas indicated in the scheme exhibit- 
ed in 10.4. The angle 613 is obtained again by inspection of the graph. The 
new reference axis Zi should pass near a cluster of points while at the same 
time the other axis Y; also should be near some other points. The variables 
represented by the first cluster of points will have high positive weights for 
the Z, factor, and the variables given by the second set of points will have 
low weights for this factor. Additional rotations may be made according to 
the outline given in 10.4. It will be evident from the following examples 
that the above procedure yields a final solution which agrees with the cri- 
teria of low complexity, level contributions of factors, and hyperplanar fit. 


1, EIGHT PHYSICAL VARIABLES 


The first illustration of a multiple-factor solution is based upon the cen- 
troid pattern of the eight physical variables, given in Table 8.4. The coeffi- 
cients in this pattern are the coordinates, with respect to the two centroid 
axes, of the eight points representing the variables. The plot of these points 
is given in Figure 10.2, in which it is apparent that the points fall into two 
distinct clusters. If two lines were passed through these clusters of points, 
they would produce excellent geometric fit to the data. Such axes, however, 
are not orthogonal and therefore not appropriate for the present method. 


The case of correlated factors will be treated in the next chapter. . 
If one axis is passed through a cluster and the other orthogonal to it, the 


standard of uncorrelated factors is met, but other standards are not well 
satisfied, "Thus if an axis M{ is passed through the first four points, the other 
axis M; will be far removed from the second cluster. The coefficients of such 


new factors would have the following properties: 
Coefficients of Mi Coefficients of Mi 


Variables s f 
1, 2,3, 4 Very high Near zero 
5,6,7,8 Fairly high High 


The first four variables would be of complexity one while the last four would 
be of complexity two. Variables 5-8 would not satisfy the criterion of low 


complexity for the present example involving only two factors. 


Fra. 10.2 Fra. 10.3 


Fie. 10.6 


Fra. 10.7 
Fias. 10.2-10.7 
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^ ES ee mA nog meet the basic standards for a multiple-factor pattern, 

iom e ES e Msare selected so as to be about equally removed from the 

Nt ee ens o Tor By inspection the resulting angle of rotation is taken 

uh 12 = —42?. Another worker, of course, might select a slightly differ- 
angle. The necessary trigonometric functions are 


cos (—42°) = .7431, sin (—42°) = —.6691. 


Substituting these values in equations (10.4), there results 


ba = .743lan — .6691aj2, 


(10.20) { 
bj, = .6691a5 + .7431a;2 , 


oni is the transformation from the old coordinates (aj aj?) to the new 
nes (ba, bj2). Equations (10.20) may also be written in the equivalent form 


(10.21) ye | .7431 .6691 
M = Ci 6691 .7431 ||’ 


mias c is the centroid pattern matrix of Table 8.4 and M is the final pat- 
ern which is presented in Table 10.1. For example, the first entry in Table 
10.1 is given by 

.830(.7431) — .396(—.6691) — 882. 


f The pattern of Table 10.1 may be examined now to see how well it con- 
orms to the standards for a multiple-factor solution. Since there are no 


TABLE 10.1 
MULTIPLE-FACTOR PATTERN FOR EIGHT 
PHYSICAL VARIABLES 
" Comaro- 
Variable Mi M: NALITY 
" M 


1. Height... eese d 
2. Arm span.....--- .922 .199 890 
3. Length of forearm .892 .171 825 
4. Length of lower leg... .861 .236 .797 
5. Weight. o. eerte te .250 .897 .867 
6. Bitrochanteric diameter .193 .790 .661 
7. Chest girth.... " 144 -127 549 
8. Chest width. . - 258 .680 529 


Contribution of factor... . 
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sampling error formulas for this type of solution, the analyst usually must 
set some arbitrary level of insignificance. The present example is based on 
a large number of observations (V = 305), and hence the coefficients which 
ordinarily might be judged as insignificant may not be so. In the analysis 
of these data by the bi-factor method, the standard error of a general-factor 
coefficient was shown to have the approximate value .066 (see p. 141). If 
this standard error were applied to the coefficients in Table 10.1, even the 
smallest value might be judged significant. While the foregoing test is not 
strictly applicable in the present case, it nevertheless throws some doubt on 
the insignificance of the small values. The multiple-factor pattern for these 
data is therefore not a good example of this type of solution. 


2. THIRTEEN PSYCHOLOGICAL TESTS 


The next example is based upon the averoid pattern, given in Table 8.9, 
for the thirteen psychological tests. The transformation to a multiple-factor 
solution in this case is made in accordance with the scheme outlined in 10.3. 
In Figure 10.3 the thirteen points are plotted in the plane of the first two 
averoid axes. The procedure in the present example differs somewhat from 
that employed in the preceding one. The first rotation is made in order to 
accomplish a leveling of the contributions of the first two factors. An angle 
42 = 50° is selected by inspection for this purpose. In such a rotation all 
points have negative projections on the second axis, designated as Yj. This 
rotation is immediately followed by the reflection of the second axis, namely, 


Y, = uit 


80 as to yield positive coordinates. Although many points will have appre- 


ciable loadings for both Y, and Y, this can be adjusted by subsequent ro- 
tations. 


The equations of transformation (10.4) may be written as follows: 


(10.22) i= 642841 + 76604, , 
Y; = —.7660Ai + .64284, , 


where it is understood that for given values of 4; and A 
coordinates with respect to Y; and Y; are obtained. This notation, in which 
the coordinates are not given explicitly, will be found convenient in specify- 


ing the axes when several successive rotations are involved. The transforma- 
tion (10.22) is followed by 


2 the corresponding 


(10.23) Y, = —Yi = .7660A, — .6428A, . 
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T xi à : 
hen, writing the resulting matrix of transformation in the form 


T, = | 5228 .7660 | 
2 = || 7660 —.6428 || ° 


e p the first two columns of Table 8.9 by Ti», yields the 
t opem of Table 10.2. The numerical calculations may be checked 
Ene stage. The new factors } 1, Ya and the factor As are mutually orthog- 

nal, and their total contribution should be the same as that of the original 


System A,, As, As. Thus, 
2.878 + 2.798 + .909 = 4.391 + 1.285 + .909 = 6.585. 
The next rotation is made in the Yi, As-plane. The plot of points is 


P in Figure 10.4, in which the coordinates are obtained from the 
rst column of Table 10.2 and the third column of Table 8.9. In this trans- 


TABLE 10.2 
INTERMEDIATE COORDINATES 


Test Fi Y: Ys 

385 .450 508 

246 .269 336 

352 .241 496 

363 .327 404 

662 .378 065 

725 .293 161 

718 .298 093 

606 .404 209 

729 .255 107 

— .048 .594 — .390 

040 .659 — .156 

— .103 .691 — .117 

145 .740 137 

Contribution 

of factor... 2.878 | 2.798 1.083 


elected. Therefore, it is impor- 


formation the first multiple-factor axis is S 
tant that this axis pass near à cluster of points and also be about 90° re- 
ts. To satisfy these requirements, the 


moved from a number of other poin 
ne of the axes is reflected again to 


angle 613 = 27° is chosen. In this case 0 
obtain positive coordinates. The final transformation in this plane may be 


Written in matrix form: 
8829 .4695 
4695 —.8829||' 


(10.24) IM, Yall = lY: Asl 
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The resulting values of M; and Y; may be recorded in the appropriate col- 
umns of Tables 10.2 and 10.3. 

The final rotation is made in the F», Y;-plane. For this transformation 
the last two multiple-factor axes are selected so as to pass as closely as pos- 
sible to clusters of points. Thus M, passes near points 10, 11, 12, and 18, 
while M; lies close to the points 1, 2, 3, and 4 when the angle of rotation is 


TABLE 10.3 
MULTIPLE-FACTOR PATTERN FOR THIRTEEN 
PSYCHOLOGICAL TESTS 
Vea | e | Seal, | Comme 
Mi M: Ms hj 
166 216 .643 488 
100 116 .414 195 
135 028 .551 323 
196 143 .500 309 
715 323 .208 659 
735 207 .263 652 
764 238 .202 681 
575 290 .350 537 
769 193 .198 668 
153 699 — .127 528 
128 668 .114 476 
— .054 682 .162 404 
091 628 .415 575 
Contribution of 
BOLO sie usu. 2.703 2.201 UGS. jussi 


* The names and brief descriptions of the tests appear in Appen. B.1. 


taken to be 04 = —23°, 


t The transformation to the new axes may then be 
indicated by 


(10.25) |M: Mj] = |Y} Yal -| 


.9205 .3907 
—.3907 .9205 || ` 


The coefficients of the factors Mz and M; are recorded in Table 10.3. The 
numerical check on the total contribution of a factor system, which can be 
made after each rotation, may again be employed on the final set of factors. 
Thus the numbers appearing in the last line of Table 10.3 sum to 6.585, 
which is the same as the total contribution of the original averoid factors. 

The multiple-factor pattern of Table 10.3 satisfies the standards listed 
above quite well, The contributions of the factors are relatively level in 
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comparison with other types of preferred solutions. The criteria of low com- 
plexity and good geometric fit also appear to be satisfied. For the present 
sample (N = 145) it is judged that a factor coefficient of two-tenths is 
insignificant. In the present solution at least four entries in each column 
and one in each row would be considered as essentially zero. The only ex- 
ception occurs for variable 8, which might be considered of complexity two 
or Possibly three if the small value .290 is considered significant. The above 
solution thus affords a good illustration of the multiple-factor type. 

. In the naming of the multiple factors, those variables having definitely 
significant weights, say, greater than four-tenths, are considered. The sub- 
groups of tests identifying the multiple factors are the same as those em- 
ployed in naming the group factors in the bi-factor solution of Table 6.13. 
The same names are then assigned to the multiple factors, as indicated in 
Table 10.3. It may be noted that each test is essentially a measure of only 
One of these factors, except Test 13. This variable appears to be a measure 
of both speed and spatial abilities. This description is reasonable inasmuch 
as the test is a measure of speed of perception of simple geometric forms. 


3. TWENTY-FOUR PSYCHOLOGICAL TESTS 
iple-factor solution is based upon the centroid 
pattern of Table 8.2 for the twenty-four psychological tests. The transfor- 
mations follow the scheme previously outlined for four factors. The dia- 
grams from which the angles of rotation are determined are given in Figures 
10.6 to 10.11. Then the successive transformations of coordinates may be 


Summarized as follows: 


The last example of a mult 


7660 -6428 

|, Fall =N Celes 660]: 7*7 790 
.1986 .6018 : 

|Z. Ysl = lY: Csi +| —.6018 — .7986]]' s = —37, 
9848 —.1737 . 

iy Ye = UZ Cp amr sep 0 1 

(10.26) 

9962 0872 -— 

|lZ, Z,|- Ys Ysl*| os 99620" —" — -5, 
8088 -4384 , 

IM; Zal = |Z: Y.l-*| —.4384  .8988|' 0, = —26°, 
j 8829 -4695 i 

|M; Mil| = |Z. Z4 l-I — 4695  .8829]' O34 = —28°. 


T 
-.4 
al 


Fra, 10.9 
Fic. 10.8 sis 


Fio. 10.11 


Fie. 10.12 Fri. 10.13 


Figs. 10.8-10.13 
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The transformed coordinates from the rotations (10.26) are presented in the 
single Table 10.4 for convenience. The axes of the last reference system have 
been designated by primes inasmuch as additional transformations will be 
made. i 

Upon inspection of the pattern involving M1, Mj, M;, and Mi in Table 
10.4, it is evident from the contribution of the first factor that it probably 
has been given too much emphasis in the successive transformations. The 


TABLE 10.4 
INTERMEDIATE COORDINATES 
Y: M, | M 
590| .144 
585) — 392| .101 
.383|— 507, .108 
-406| 496| 1083 
1466) — agel .083 
sl- 149| .238 
i 127| .095 
Hole 280| 093 
eb 181 307 
p — 1236] .022 
= Z = ¿091| .175 
Z 02 069|— :096 
= SI 266|— .089 
teal — [059| 1442 
-238 058|  .468 
P 331|  .409 
NU. 1 — .053|  .504 
A i 195|  .447 
Wu i 151| .347 
Aht 399| 1257 
AE > 296| :161 
eal 2 340| 1373 
As i 474|  .197 
lS 3 073 222 
1123|— . 
Contribution of adh adl 
factor... as 2.487 


contribution of M1 is about twice that of Mj or Mj. In Maga obtain a 
more level distribution of the total variance, and a solution wit oe ims 
appreciable weights for the first two factors and larger bc is : he rd 
ables identifying the last two factors, two co p res pem * 
The twenty-four points are plotted in = pienia ii n t f : t ti 

Figures 10.12 and 10.13, respectively. By selecting asma E M rota in 
in each case, the desired adjustments are made. These supplementary rota- 


tions may be indicated as follows: 


„|| 9848 za —" 
jan, Mil| = IMi Mall - || 1737 98481" 
(10.27) 9848 .1737 —" 


, 24 


|M; Mil| = IM; Mill | —.1737 .9848 
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The final multiple-factor pattern is presented in Table 10.5. The checks 
on the calculation of the coefficients in Tables 10.4 and 10.5 after each rota- 
tion are summarized in Table 10.6. The sets of four factors comprising an 
orthogonal system are indicated, and their total contribution is compared 
with that of the original four centroid factors. Since the contributions of the 


TABLE 10.5 


MULTIPLE-FACTOR PATTERN FOR TWENTY-FOUR 
PSYCHOLOGICAL TESTS 


Verbal | Speed | Deduce- | Memory ae 
Mi M: Ma Me Aj 
.098 .324 .616 .203 .535 
.068 .153 .410 .130 .213 
.095 .119 .532 .181 .323 
.150 177 .530 | .115 .948 
.746 .150 .255 .154 .668 
.722 .051 .279 .251 .665 
.807 .085 .271 .111 .744 
.535 .258 .379 .140 .516 
797 — .039 .287 .305 .750 
.277 .664 | — .191 .139 .573 
.207 .612 | — .045 .286 .530 
.128 -717 .091 .029 .540 
.239 .634 -312 .021 .557 
.281 .188 | — .019 .482 321 
.106 .138 .078 | .500 .286 
.053 .218 .945 | .454 .375 
.153 .245 | — .027 .616 .464 
.006 .386 .199 .522 .461 
.123 .223 -175 | .392 .249 
.307 177 .459 . 292 .422 
.172 .460 331 .245 .411 
.311 »AM .400 .399 .430 
.314 .291 .537 .251 .535 
.387 .457 .142 .306 .472 
Contribution 
of factor. .| 3.430 2.917 | 2.683|2.358 |........ 


* The names and brief descriptions of the tests appear in Appen. B.1. 


respective Systems agree with the number 11.383, except for errors due to 
rounding, it may safely be assumed that the calculations are accurate. 
Using the same level for judging significance of factor coefficients as in 
the preceding example, it is clear that the solution of Table 10.5 is a good 
one. There are at least nine insignificant entries in each column, indicating 
good geometric fit, Each column also has a sufficient number of definitely 
significant weights for the naming of the factors. From the descriptions of 
the variables with large coefficients for M;, M. 2, and M ,, these factors may 


uA — ——— T 


ORTHOGONAL TRANSFORMATIONS 233 


be appropriately named “‘verbal,” "speed," and “memory,” respectively. 
The third factor has appreciable loadings for Tests 1, 2, 3, 4, 20, 22, and 23. 
The first four of these have been called "spatial" tests and involve the de- 
duction of relations among geometric objects. The last three tests involve 
logical and arithmetical relations. The common element of all these tests 
appears to be the deduction of relations, regardless of the specific content 


of the respective tests. The factor M; might then be named “deduction.” 


TABLE 10.6 
TOTAL CONTRIBUTIONS OF SYSTEMS OF FACTORS 
OOC | Ya¥2CsCu | aYme MiY:YaYa | M;Z:ZaYı | MiAGZ74 MiMiM;M; | MM:M:Mı 
11.383 | 11.384 | e—a [D^ PNE 
-383 11.384 11.385 11.383 11.383 11.383 | 11.382 11.388 


e 10.5 for the first thirteen tests may be com- 
Je 10.3 for the same tests. A remarkably close 
Even in the case of the third factor, which 
utions, there is & close agreement for 
The first four tests, which were em- 
able 10.3, have very similar weights 


The factor pattern of Tabl 
pared with the pattern of Tab 
Agreement is evident in general. 
is given a different name in the two sol 
the variables common to the two sets.- 


ployed in the naming of this factor in T 1 g 
in the second solution. As pointed out above, however, the high loadings 


on this factor for the three additional tests suggested a somewhat different 
name in the latter solution. This illustrates the principle, stated in 5.8, that 
a factor solution cannot be considered as independent of the set of variables 
employed. Thus the larger battery of tests leads to a different name for the 
third factor and to the inclusion of a “memory” factor which is not present 
in the smaller set. The agreement of the common portions of Tables 10.3 and 
10.5 is evidence of the stability of factor patterns when & particular form of 
solution has been selected. Similar stability was indicated in 6.11 for the bi- 


factor form of solution. 


CHAPTER XI 
OBLIQUE SOLUTION 
11.1. Introduction 


In Chapter V.a number of preferred types of solutions involving uncorre- 
lated factors were introduced. Methods for the calculation of such factor 
patterns were developed in subsequent chapters. Now the assumption of 
uncorrelated factors will be discarded, and the procedures leading to oblique 
solutions will be considered. It is clear that a certain simplicity of interpre- 
tation is sacrificed upon relinquishing the standard of orthogonality. This 
disadvantage may be ofiset, however, if the linear descriptions of the vari- 
ables in terms of correlated factors can be made simpler than in the case of 
uncorrelated ones. Generally this is possible. Hence the preferred type of 
oblique pattern which is proposed is one which approximates the uni-fac- 
tor form. 

The method of analysis of this chapter consists in the rotation of some 
itial orthogonal pattern to an oblique solution.* The communalities of the 
variables, which are determined by the common-factor space of the prelimi- 
nary solution, remain invariant under this transformation. Hence the 
uniqueness for the oblique solution will be the same as that of the original. 
Therefore, the entire development will be made in the common-factor space. 

The geometric setting for the oblique form of solution is presented in 
11.2, where the distinction between pattern and structure is again made. 
This is followed by a detailed outline of procedure illustrated by a simple 
example. Then several additional numerical illustrations are given in 11.4. 
In the final section an illustration is given of the inapplicability of the 
oblique form of solution for variables which lack a clear design. It will be 
shown, however, that a suitable oblique uni-factor pattern can generally be 
achieved when there is evidence of grouping or clustering of the variables 


in 


11.2. Geometric Basis for an Oblique Solution 

In 2.4 the definitions of factor patterns and structures were formulated. 
It was also pointed out that, when the factors are uncorrelated, these con- 
cepts become identical. In the foregoing chapters, therefore, no distinction 
was necessary, and the term “pattern” was used synonymously with ‘‘solu- 
ton." When correlated factors are employed, however, the solution con- 


* For an equivalent met 


hod which obviates rot 
“A Simple Method of F. 


ation of axes see Karl J. Holzinger, 
actor Analysis," P; 


sychometrika, IX (December, 1944), 257-62. 
234 
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sists of two essential parts, i.e., the pattern and the structure. Now this 
distinction will be shown geometrically. 

À pattern, in terms of common factors only, may be represented as fol- 
lows: 


zi! = bal + bole + sil dinl nm (j—51,2,...,mo), 


where 6’s are employed to denote coefficients of correlated factors. As point- 
ed out in 3.8, the double prime denotes a variable projected into the com- 
mon-factor space. Since the analysis of this chapter is entirely in the 
common-factor space, the primes will be dropped for simplicity, and the 
foregoing equation written 


(11.1) zi = bafi + bjf t+... + dink mn G712,....,0). 


The coefficients may be considered as the coordinates of a point z; with re- 
spect to the factor axes. This interpretation may be made whether the fac- 
tors are represented by orthogonal or oblique axes. In the latter case the 


theory of 3.6 is employed. 


Firg. 11.1 
c ideas may be illustrated by Figure 11.1. 
ors F, and F» is known, the unit vec- 
ted by an angle 012 = arc COS rr,z,. 
i i Igebraic signs of the 
i thu mined. The alg: r 
on peu the same as in the conventional orthog- 


For the case of two factors thes 
When the correlation between the fact 
tors representing F, and P; are separa 

s deter 
Coordir i “quadrants” are 
onal amg ed variable z; is represe nde desi ide eerie 
direction are determined by its coordinates. These € ^y es are d 
to be the coefficients of the respective factors m the analytic expressio 


the variable " 
(11.2) z; = bali + bia? G 32, 
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From the definition of general Cartesian coordinates, given in 3.6, it may 
be noted that the coordinates bj, b;» are given by the line segments OP and 
OQ, respectively. For the hypothetical variable z;, the first coordinate is 
positive and greater than unity, while the second coordinate is negative and 
less than unity. 

The length of the vector z; can be determined by means of formula (3.32), 
which in this case may be written as follows: 


2 2 


D'(02;) = bj,b;, cos 6, 
(11.3) : >> “a 


= bibi Cos O11 + bjrbj2 COS O12 + bj2bj1 COS 05 + bj2b 2 COS 022 - 


In this formula each of the angles 61: and 622 is equal to zero, and 612 is the 
angle between the reference axes. Hence cos 6; = cos b22 = Land cos 6: = 
Tr,r,. The expression (11.3) then reduces to 


(11.4) D'(02;) = bj, + bha + 2babisre p, « 


The right-hand member is the communality A? of the variable Zj, the pro- + 
jection of which is represented in the common-factor space by (11.2). The 


length of the vector is then equal to the square root of the communality, 
that is, 


(11.5) D(0z) = h;. 


The geometric interpretation of the correlation of a variable with a fac- 
tor will now be given. Let the angle between th 
ence vector F; be denoted by ¢. Also let the 
of the vector z; upon the F, and F, axes be 


e véctor z; and the refer- 
2e projections of the end point 
given by M and N, respectively, 


ei indicated in Figure 11.1. From the right triangle OM 2;, it is apparent 
that 
= DOM) 
This formula reduces to 
(11.6) D(OM) = h; cos ¢ 


upon making use of (11.5). Formula (11.6) may be simplified 

f i further b; 
applying (3.51), which for variables 2; and F, Pm SENE 
(11.7) 


Tir, = hihr, cos ¢. 
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Since the length, or “eommunality,” of any factor is unity, this expression 
may be written in the form 
(11.8) Tir, = hj cos 6. 


s of correlations of variables 
plicity. Substituting (11.8) 
ference axis may finally be 


I i A s 
a formula, and in all other representation: 
int factors, the prime will be dropped for sim 

' o (11.6), the projection of a vector upon à re 
expressed as follows: 


ma D(OM) = ri, - 


projection, D(ON), of the vec- 
iable with the second factor. 
en by the projection 


ira iar manner it can be shown that the 

tiem the P; axis is the correlation of the var i 

of eit rse, the correlation between two factors 1$ also giv 
ither reference vector upon the other. 

a : referr ing to Figure 11.1, the distinction between a coordinate and a 

2 elation can clearly be seen. The coordinates may be positive or nega- 

a and may be greater than one. A correlation coefficient also may be 

T mcs or negative but can never exceed unity. It may also be — 

Vect e coordinates and correlations approach coincidence as the reference 

ors approach orthogonality. 


"s complete solution involving corre TS c : 
aà structure. The factor pattern may be exhibited as in equation (11.1) 


9r, more compaetly, in à table giving the coefficients of the factors. The 


Structure may be presented in tabular or matrix form. For the preceding 
bles the structure may be 


lated factors must consist of a pattern 


h : ` 
= Pothetical example of two factors and n varia 
Corded as follows: 
Tir, TIF, 
Tor, TF. 
Tyr, ifi 
Tag, TFs 
In addition to the pattern and structure the oblique solution should also 
tors. 


In 

clude a table of intercorrelations of fac 

lı ; 
3. Procedure for Obtaining an Obligue Solution 


Becca 
limi n oblique solution can be 0 


i 
nary orthogonal pattern Suc 


btained by means of à rotation of some pre- 
h as the bi-factor, centroid, OT averoid. In 
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this sense, the method of analysis is similar to that employed in the last 
two chapters. Unlike the procedures employed there, however, the angles 
between the reference vectors in the present case are permitted to take on 
values other than 90°. 

A complete theoretical development, employing literal notation through- 
out, would be unnecessarily complex and tend to conceal the actual sim- 
plicity of the procedure. For clarity, then, the development will be applied 
to the example of eight physical variables and may be readily generalized 
to larger sets of variables. The various stages of analysis are presented in 


outline form, including a detailed description of method in each step of the 
procedure. 


1. INITIAL ORTHOGONAL PATTERN 


As pointed out in 2.7 it is advisable in factor analysis to select the vari- 
ables according to a design which is later verified or modified by the factor 
solution. If the resulting pattern reveals distinct subsets of variables, it 
will be especially effective for a clear interpretation of the oblique solution. 
Such a preliminary pattern sometimes may arise without an explicit formu- 
lation of a design. In either case, a pattern furnishing evidence of subsets 
will be regarded as “efficient.” j 

Geometrically, the points representing a set of variables which have been 
analyzed into an efficient orthogonal pattern will form distinct clusters. 
For such a configuration new reference vectors can be taken through these 
clusters so as to meet the standard of linea; 
new reference system will be oblique. 


If an orthogonal solution for a given set of variables is efficient, then any 
other orthogonal pattern based upon these data is also efficient. The first 
step in obtaining an oblique solution is, then, to select any such initial pat- 
tern. In the present development the centroid solution of the eight physical 


variables, given in Table 8.4, is selected arbitrarily as the initial form. This 
pattern is repeated here for convenience, 


r fit.* In general, of course, the 


2. SUBSETS OF VARIABLES 


fundamental groups, and then the 
tion. 


* See Standard 5a of Sec. 5.2. t See 2.7. 
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For the eight physical variables the B-coefficients were determined in 
4.6 and have the values 


B(1, 2, 3, 4) = 235,  B(5,6,7,8)- 179, 


indicating two distinct subgroups. By noting the signs of the coefficients in 
Table 11.1, it is clear that the vectors representing the first four variables 
form a cluster in the fourth quadrant, while those representing the last four 
variables form another cluster in the first quadrant. The above grouping of 
variables is thus verified. 


TABLE 11.1 


INITIAL CENTROID PATTERN OF EIGHT 
PHYSICAL VARIABLES 


COEFFICIENTS OF 
Common FACTORS 


VARIABLE 
aji aj? 
pa bees 
1. Height..... | .830 | — .396 
2. Arm span.....-+ .818 — .469 
3. Length of forearm s .777 | — .470 
4. Length of lower leg. - - . .798 | — .401 
5. Weight.. .786 .500 
6. Bitrochan .458 
7. Chest girth. .444 
8. Chest width 333 
Sum... £ — .001 
Contribution of factor.... 4.439 1.526 


iitial pattern involving only two or three common fac- 
tors, the clusters can be seen directly from the plot of points with respect to 
the originalaxes. The eight variables, whose coordinates are given in Table 
11.1, are plotted in Figure 11.2, where the two distinct clusters are clearly 
evident. Although the figure is very simple for two factors, the geometric 
representation for many factors becomes exceedingly complex. Then the 
methods of B-coefficients and inspection of the coefficients in the pattern 


must be employed. 


In the case of an ir 


3. DIRECTION COSINES OF OBLIQUE REFERENCE AXES 
The next stage in the analysis is to determine the directions of the oblique 
by inspection, from the origin 


Teference vectors. Lines may be drawn, 
through the clusters. The angles which these new axes make with the old 


may be measured, and the transformation may be determined thereby. 
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"This procedure is rather subjective, however, and will be replaced by a more 
objective analytic method. 

The average, or centroid, of each cluster of variables can be determined 
from the coordinates of the corresponding subset of points. These values 


y from the initial factor pattern. For the illustra- 
he cluster 1, 2, 3, 4 is the point whose coordi- 
:4340), while that of the cluster 5, 6, 7, 8 is 
-2 the new oblique reference axes are drawn 


nates are given by (.8058, — 
(.6748, .4337). In Figure 11 


ors. Instead, the coordinates of these points 


initial patt indi S 
through them can be imagined. a EE bons dud 


OBLIQUE SOLUTION 241 


. Knowing the coordinates of a point on a line through the origin, the direc- 
tion cosines of that line can be determined. As indicated in 3.5, the direc- 
tion cosines are given by the ratios of the respective coordinates to the dis- 
tance of the point from the origin. The distance of the first point from the 


origin is 

V/(.8058)? + (—.4340)? = .9152. 
Then the direction cosines of the line through this point, with respect to 
the C; and C; axes, are given by 


—.484 
8058 _ 9995, An = DEA = — 4722. 


M = “9152 7 


These are then the direction cosines of the new axis yı with respect to the 
original axes, In a similar manner, the direction cosines of the y; axis are 
found to be 
Ma = S412, ov = -5406. 

obtaining these direction cosines will 
he-centroid of a cluster, the point cor- 
bset of variables is used. Let the 


Now an alternative approach for 
be indicated. Instead of employing t 
responding to the composite* of this su 
composite variables be defined by 
Laud apie es 


(11.10) n-nbatada 


The standard deviations of these variables may be computed by means of 


(2.40), as follows: 
| an = VEF 2018) = 3.7465, 


11, 
TM „ = V4 + 28.820 = 3.4117. 


O: 
site variables with the C; and C; factors 
la (2.43). In this case, the zo of the 
lation of the first composite variable 


Then the correlations of the compo 
can be calculated by means of formu 
Ormula is taken as a factor. The corre 
With C; is thus given by 

930 + 818 + -777 + 798 _ 8603, 
e 3.7465 


Tuc, 
n from Table 11.1. Since the 


are take s i 
^ he coefficient of Ci in 


Where the values in the numerato n 
his value 1$ also t 


centroid factors are uncorrelated, t 
* See 2.10. 
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the linear expression for the standardized composite variable. In a similar 
manner the remaining correlations, or coefficients, can be obtained and ar- 
ranged in the form of a reduced pattern as follows: 


u = x = .8603C; — 4634€, , 
(11.12) 
us A = 7911C, + .5085C; . 


Such pattern equations have the same properties as those for individual 
variables. n 

The coefficients of the reduced pattern may be interpreted as the coordi- 
nates of two points representing w and us. The direction cosines of the 
lines from the origin through these points can be caleulated as before by 


dividing the coordinates by the appropriate distances. These values are 
found to be 


Au = 8804, A4 = —.4742, 
At) I = 8412, N= 5407. 


The equivalence of the direction cosines of yı and ys with respect to Ci and 
C: obtained by the two methods is proved in Appendix B.10. 

The method of reduced patterns will be employed in the remaining ex- 
amples. Although the arithmetic is somewhat more involved than in the 
method of averages, the reduced pattern furnishes additional interpreta- 
tions of the variables and factors. The correlations of reduced variables 
with factors are also employed for the estimation of factors in the next 


chapter. 
4. INTERCORRELATIONS OF FACTORS 


After the direction cosines of the oblique reference vectors have been ob- 
tained, the correlations between these factors ean be determined. By em- 
ploying (3.41) and (3.29), the correlation between y, and yz in the present 


example is found to be 
Try, = .8804(.8412) — .4742(.5407) = .4842. 


The self-correlations, 


or variances, of the factors may be calculated in the 
same manner. Thus, 


95, = (8804)? + (— 4742) 
v5 = (8412) + (. .5407)2 


1.0000 , 
1.0000 . 
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These calculations serve as a check on the values (11.13) because the sum 
of squares of the direction cosines of any line must be unity. From the cor- 
relation between factors the angle of separation of the reference vectors can 
readily be determined. In the present case this angle is given by 


Oy. = arc cos .4842 = 61°2’. 


5. FACTOR STRUCTURE 

The projections of the vectors representing the variables upon the oblique 
reference axes can now be determined.* As indicated in (11.9), such projec- 
tions are the correlations of the variables with the factors, i.e., the elements 


Fig. 11.3 


of the factor structure. In order to indicate clearly the derivation of the 
formulas for the structure values, Figure 11.3 has m verd t 2 
figure the oblique reference vectors Yi and 72 sp ius venia a E 
quadrant of the Cı, Ce reference system. ‘This has been, done mere y 
rdinates of the points with respect to the oblique ref- 
quom of the original coordinates. Since, in factor 
d the projections (i.e., the coefficients and the correla- 


is suggested as the simplest and the one best 
proach is elections are obtained, and then in the next 


* It is possible to obtain the c 
erence system directly by trans 
analysis, both the coordinates An 
tions) are desired, the present 8P. 
adapted to systematic calculations. First the pr! j 
stage of the analysis the coordinates are calculated. 
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clarify the development, the results being the same even if the oblique axes 
fall in other quadrants. The angles from the C; axis to the y; and yz axes 
are denoted by a and £, respectively. 

Any variable z; may be represented in this figure by a point whose co- 
ordinates (aj, aj?) with respect to the original centroid axes are the coeffi- 
cients in the initial pattern equation. The variable may also be represented 
by the vector from the origin to the point z;; (aj, aja). The angle from the 
C; axis to this vector is denoted by $. Then the projection of the vector 2; 
upon the y; axis is given by 


(11.14) D(0M) = D(0z;) cos (6 — a). 


As already noted, the projection D(OM) is equivalent to the correlation 
Tiy, and the length of the vector is D(0z;) = h;. Making these substitu- 
tions in (11.14), and expanding the cosine of the difference of two angles, 


this formula becomes 
Tiy, = hj [cos ¢ cos a + sin $ sin a], 
= (hj cos ¢) cos a + (h; sin $) sin a. 
Now, h; cos ó and h; sin ¢ are the projections aj, and aj; of the vector 2; 0n 
the C; and C; axes, respectively. Then the formula finally becomes 


(11.15) Tj, = Qj COS a + aj; sin a. 


In a similar manner it can be shown that the projection, D(ON), of the 
vector z; on the yz axis, that is, the correlation T jy) is given by 


(11.16) Tj, = aj cos B + aj sin B. 


The results of (11.15) and (11.16) may be summarized in the following ma- 
trix form: 


(11.17) [rs r&b = lan aal -| 


An — cosa Xi = cos B 
An — sina 25-sinf|' 


where, in the last matrix, the elements of the first column are the direction 
cosines of yı with respect to C, and C», and those of the second column are 
the direction cosines of yo. Although (11.17) was developed on the basis of 
Figure 11.3, this expression is true for different positions of the new axes 71 
and yə, as, for example, that indicated in Figure 11.2. Written in the form 


(11.17), the determination of the elements of a structure can be generalized 
to problems involving more than two factors. 
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For the example the direction cosines are given in (11.13) and may be 
put in.a matrix as follows: 


(11.18) 


| .8804 .8412 
—.4742 .5407||' 


Then multiplying the centroid pattern matrix of Table 11.1 by this matrix, 
the structure values of Table 11.2 are obtained. The correlations of the 
composite variables with the oblique factors are obtained by multiplying 
the reduced pattern matrix of (11.12) by the matrix (11.18). These values 
are presented in the reduced structure of Table 11.2. 


TABLE 11.2 
OBLIQUE SOLUTION FOR EIGHT 
PHYSICAL VARIABLES 


STRUCTURE PATTERN 
VARIABLE 
jy, js, Lankiness | Stockiness 
j - * n Ye 
919 484 .894 051 
943 435 .956 —.027 
907 399 .932 —.052 
893 454 .879 029 
455 932 .005 930 | 
374 813 —.025 825 
312 740 —.060 769 
412 724 .080 685 
A 
Reduced Reduced 
Structure Pattern 
Bs ae Fe BE .977 473 .977 .000 
[E .455 .940 .000 .940 


A check on the calculation of the oblique factor structure can be made by 
changing the order of the multiplications and additions in the calculations 
of the sums of the structure values. For two factors this may be written in 


the form 

(11.19) Daj + AZ; = Erg, ((—1,2), 
trary but the same for all three sums. To illus- 
s check, consider the correlations of the eight 
1to8in (11.19). The check 


Where the range of j is arbi 
trate the application of thi 
variables with yı. Thent = 1 and j ranges from 
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on the first column of Table 11.2 then consists in the agreement of the sum 
of this column with the evaluation of the left-hand member of (11.19). 
This sum is 
8 
> te, = 5315, 
i=l 
while 


8 8 
An D ai + Ai D aj = .8804(5.922) — .4742(—.001) = 5214. 


j=1 j=1 


Since the discrepancy is evidently due only to rounding, the calculations of 
this column of the factor structure are accepted as accurate. Of course, the 
check might have been applied to the entire first column, including the 
composite variables or, again, to the calculations of the correlations of the 


composite variables alone. Similarly, the second column of the factor struc- 
ture is checked. 


6. FACTOR PATTERN r 
^ a E 


.. In order to have a complete solution in terms of correlated factors, the 

linear descriptions of the ine are required as well as their correlations 

^. with the factors. The pattern coefficients are the coordinates with respect 

to the oblique axes of the points representing the variables and could have 

been obtained directly* from the initial factor pattern. It is more conven- 

ient, however, to calculate these values after the oblique factor structure 

has been obtained by means of the relationship between the pattern and 
structure which is described in Appendix B.11. 

If the oblique factor pattern for the illustrative example is denoted by 


(11.20) 


Zi = bay, + biy, (j= 1,2,...,8)5 


the problem is to determine the coefficients bi, bja. For any variable Z; 


multiply (11.20) by yı and Yz in turn, sum for the N values, and divide by 
N. The resulting equations are 


(11.21) ( Tm Sba bares, 
Tm = dary, + bj. 


There is such a pair of simultaneous equations for the determination of the 
two unknowns b; bja for each variable z;. In equations (11.21) the terms 
*See n.*, p. 243. 


LS 
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Tiy, and rj, are known from the factor structure, and the correlation be- 
tween the factors is known from the fourth step in the analysis. The de- 
terminant of coefficients of the unknowns by, bj» is 


1 4842 1 
4842 1 


and remains the same for successive calculations of the b's. The determina- 

- tion of the factor coefficients may be caleulated by the method of determi- 
“nants, but, especially when many factors are involved, a more efficient pro- 
cedure is desired. Such a procedure is described in Appendix G.3, where the 
present example is used as an illustration. The resulting values of the fac- 
tor coefficients appear in the pattern matrix in Table 11.2. In this table the 


B" oblique factor pattern is also presented. 


7. CONTRIBUTIONS OF OBLIQUE FACTORS 


After an oblique factor pattern has been obtained, the direct and joint 
contributions of these factors can be determined. The communality of a 
variable z;, as given by (11.1), may be expressed as follows: 


; a 
ap M = bh tbh to F Oat Dabaran e e 
.22 
) € E 2b; mint Fp iE . & 
"ue. 
The direct contributions of the factors to 2; are: iven by the first m terms 
the factors are furnished . 


in this expression, while the joint contributions of the 
by the remaining terms. The total direct contributions of the factors are 
obtained by summing the direct contributions of the respective factors for 


all the variables as follows 


(11.23) p» Sas ZHe 
^ Jel j=1 


jml 


The total joint contributions of pairs of factors are given by 


" n n 

(11.24) 2rr p, >, babe " 2rg, e, >, babis pieta Bre, Fn >, bimbi i 
i ger des 

arranged conveniently in a triangular 

tions are put in the diagonal and the 


he triangle. 


"These two sets of expressions can be 
matrix in which the direct contribu 
joint contributions in the remainder of t 
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In the illustrative example the total direct contributions of the two fac- 
tors are given by 


p 


= .894* + .956? + . . . + .080? = 3.365, 


~ 
V 
S 

| 


= .051° + (—.027)? +... + .685? = 2.613, 
while the total joint contribution of these factors is 


8 
2r, 2 bibis 


j=1 
= 2(.4842)[.894(.051) + .956(—.027) +... + .080(.685)] = —.010. 
The contributions of the respective factors may then be arranged in the form 


of Table 11.3. The grand total of the contributions of a set of oblique eed 
tors should, of course, be equal to the total communality of the origina 


TABLE 11.3 


TOTAL CONTRIBUTIONS 
OF FACTORS 


Grand total =5.968 


solution, In the present exam 
5.965, which is the sum of the 
tributions of the factors accou 
common-factor variance, 


ple this total agrees closely with the value 
contributions in Table 11.1. The direct oor’ 
nt for all but a negligible amount of the tota 


11.4. Numerical Ill ustrations 


be presented, employing the psychologi- 
be based upon initial centroid patterns, 
-four variables. The other two solutions 
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It will also be possible to compare the oblique solutions, for a given set of 
variables, based on two different initial patterns. 

Since the same data are employed in all four solutions, the determination 
of the subsets of variables may be indicated now. According to the method 
of B-coefficients of 2.9, the twenty-four psychological tests are grouped into 


the following composites: 


vy =a +z: +2: +2, 
v) = z +z +z +2 +2, 
(11.25) v3 = Zio + Zn + 22 +2, 
V4 = Zu + 215 + as + 417 + ns + 219, 
Ug = 220 + Za + 222 + 223 + 22. 
riables are employed, these are grouped into 


t with the first three composites of 
h case by inspection of the initial 


When only the first thirteen va 
three subsets which are in agreemen 
(11.25). This grouping is verified in eac 
factor pattern. 


1. SOLUTIONS BASED UPON INITIAL CENTROID PATTERNS 
* 


The first illustration is based upon the initial centroid pattern of Table 
8.1. The grouping of the thirteen variables into the three composites, vi, t», 
and vs, is substantiated by the preliminary solution. These composites are 
then employed in the determination of the direction cosines of the oblique 


TABLE 11.4 
REDUCED CENTROID PATTERN FOR THIRTEEN 
PsYCHOLOGICAL TESTS 
Variable | Cı | C: | ei D(0uj) 
.6535 .0735 | —.5012 .8268 
.8448 .3759 . 2330 .9536 
.6841 — .5697 1405 .9013 


reference axes. The standard deviations of the variables vı, v», and vs have 
been computed in 2.11 and are given by 7s 7 2.843, ov, = 4.214, and o», = 
3.147, The reduced centroid pattern may then be calculated and written 

te variables, which are in stand- 


as in Table 11.4. In this table the composi ; 
ard form, are designated by u's; and the distances of the points, represent- 


ing these variables, from the origin are given in the last column. The direc- 
tion cosines of the oblique reference axes Y Y» and ys (which are the lines 
from the origin through the points representing the composite variables) are 
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obtained by dividing the elements (coordinates) in the rows of Table 114 
by the respective distances. The resulting values can be recorded in matrix 
form as follows: 


^a Deo Aes 
Ci -7904 .8859 -7590 
(11.26) C5 .0889 .3942 —.6321 


C3|| —.6062 .2443 .1559 


: In this matrix the direction cosines of the respective oblique axes with re- 


spect to the centroid axes are recorded in columns. 

The correlation between any two oblique factors is obtained simply by 
summing the paired products of corresponding direction cosines of the ref- 
erence axes representing them. In like manner, the variances of the fac- 
tors can be obtained by taking the sum of the squares of the respective 
columns of (11.26). Of course, these variances must be equal to unity, and 
hence they can serve as a check on the preceding calculations. The corre- 
lations among the factors are presented in Table 11.5. If it is desired, the 


TABLE 11.5 
INTERCORRELATIONS OF FACTORS © 


LE 
Er gu es or RR p 
Ke mas ER a .587 | 1.000 |. * 
Wis Sis eere 449 | .461 | 1.000 


angles between these oblique axes can be determined f rom the correlations. 
In the succeeding analysis, however, the actual values of these angles are 
not required, but only their cosines, i.e., the correlations. 

The oblique factor structure can be determined as indicated by (11.17). 
Upon multiplying the initial pattern matrix of Table 8.1 by the matrix of 
direction cosines (11.26), the factor structure of Table 11.6 is obtained. 
Similarly, the reduced structure is found upon multiplying the reduced pat- 
tern matrix of Table 11.4 by the matrix of (11.26). The calculation of the 
Structure of Table 11.6 can be checked by the application of formula 
(11.19) extended to three factors. 

From the intercorrelations of the factors and the structure, the factor 
pattern can be determined. Employing the method of G.3, the oblique fac- 
tor pattern for the thirteen tests and the reduced pattern are calculated and 
are presented in Table 11.6 also. The total direct and joint contributions of 
the three oblique factors are given in Table 11.7. The grand total of the 
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nae anyon differs from the total original communality only by 0.015, so 
at the entire oblique solution may be said to check. 


TABLE 11.6* 
OBLIQUE SOLUTION FOR THIRTEEN 
PSYCHOLOGICAL TESTS 
STRUCTURE PATTERN 
Test 
Spatial , 
Tiy, Tin rjy, | Relations Verbal | Speed 
J om Yi và 

loe .748 .406 .430 .731 | —.089 .142 
2.. .445 .264 .204 .441 .004 .004 
p MP (604 | .324] -157 {721 | —.090 | —.142 
4... “559 | .386 | -266 .b08 .090 | —.003 
[m .451 .807 .429 | —.058 .801 .087 
6... .489 .807 .339 .037 .809 | —.051 
dos “447 | .846| -355 | — .068 .901 | —.030 
8. “537 | .717 | -420 .155 .591 .078 
9.......| .435 .841 “311 | —-068 .919 | —.081 
19.721 10758 | .3H| 712 — .385 .164 .809 
Les .302 .957 ‘677 | — -039 .077 .659 
12.. .316 .222 .124 .073 | — -177 -773 
13 .582 .419 .123 .351 | —.061 .594 
ee [LEE 

Reduced " Reduced 

Structure ¢ Pattern 

in m 


* Based upon centroid solution of Table 8.1. 


TABLE 11.7 


TOTAL CONTRIBUTIONS OF FACTORS 


Grand total =6.981 


The second illustration is based upon the initial centroid pattern of Table 
8.2 for the twenty-four psychological tests. Since this preliminary solution 
involves only four factors, the common-factor space for the succeeding 
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oblique solution consists of four dimensions, at most, according to Theo- 
rem 3.5. Hence only four composite variables are desired, inasmuch as each 
of these variables determines a direction for one of the oblique axes. Of the 
composites (11.25) determined by B-coefficients, the first four agree most 
favorably with the centroid solution of Table 8.2. The points representing 
the variables of each of these subsets fall into distinct *sedecimants"* with 
only two minor exceptions, as may be indicated symbolically by 


he MCh, T Vy 

ve: CF, 7, 7, +), 

tas (+, +, —, —), 

vit (Ay bys). 
On the other hand, the points representing the variables of the last subset 
do not fall into a single "sedecimant" but overlap with the preceding sub- 
sets. Thus, from inspection of Table 8.2, variables 20 and 23 might be 
grouped with those of vı, variables 21 and 24 with those of vs, and variable 22 
might be included with those of v4. For simplicity, however, these variables 
are not regrouped, and the first four composites of (11.25) are accepted. It 


TABLE 11.8 
REDUCED CENTROID PATTERN FOR TWENTY-FOUR 
PSYCHOLOGICAL TESTS 
Variable CO Cs [^ Ci D(Ou;) 
[T E .6627 — .2343 .9429 | —.2547 .8225 
Aa...ssss| . 7848 —.8935 | —.3272 .1652 .9514 
Us waaa .6746 .4264 | —.2609 | —.3143 .8965 
ME .7257 .3029 .2261 .2930 .8691 


is to be expected that the last five tests will be expressed, in the final oblique 
pattern, in terms of the same factors which are determined b 
variables most nearly related to them. 

In order to calculate the reduced centroid pattern, the standard devia- 
tions of the composite variables are required. These were obtained in 2.11, 
and the first three were employed in the previous example. The standard 
deviation of the fourth composite variable is Tv, = 3.945. The elements of 
the reduced pattern are obtained by dividing the sums of coefficients of the 
variables comprising a composite by the corresponding standard deviation. 
The reduced pattern is presented in Table 11.8. The direction cosines of 


* A “sedecimant” is one of the sixteen regions into 
the four reference axes. These regions are analo, 
and the eight octants in ordinary Space. 


y the composite 


which the four-space is divided by 
gous to the four quadrants in the plane, 
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the oblique reference axes yi, Ys; Y» and y, with respect to the original frame 
of reference are given in the columns of the following matrix: 


^a 2 Aas Yad 

Ci .8057  .8249  .7525 .8350 

(11.27) C: || —.2849 —.4136  .4756 .3485 
Cs .4169 —.3439 —.2910 .2602 


C,|—.3097  .1736 —.3506 3371 


among the four oblique factors are obtained as before 
11.9. The complete oblique solution, including 
for the twenty-four tests is given in Table 11.10. 
n, the contributions of factors are presented 
ees with the total original communality 
lof this-table indicate the almost level 


The correlations 
and are presented in Table 
both structure and pattern, 
As a final check on the solutio: 
in Table 11.11. The grand total agr 
(11.383). The values in the diagona 
contributions of the four oblique factors. 


TABLE 11.9 
INTERCORRELATIONS OF FACTORS 


1.000 |... 
.463 | 1.000 |......-- 
514 .600 


n are distinct in an oblique solution, the 
question arises as to how to employ these matrices in the naming of the 
factors. All the elements of the structure are generally different from zero 
because the factors are correlated. This may be seen geometrically from 
the fact that the vectors of the variables usually have appreciable projec- 
tions on all the reference axes. This is true even if à particular vector lies 
directly on one of the reference axes. Thus it appears that the structure 18 
not very useful for the naming of factors. On the other hand, when a vec- 
tor lies on one of the reference axes, its end point has zero coordinates with 
respect to the remaining axes. Thus the points representing the variables 
of a subset (lying close to and determining one of the oblique axes) have 
large coordinates with respect to this axis and small coordinates with Te- 
spect to the remaining reference axes. The oblique factor pattern, consist- 
ing of such coordinates, thus approximates the uni-factor form and furnishes 
a basis for clearly identifying the factors. In the case of composite variables 


Since the structure and patter 
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TABLE 11.10" 
OBLIQUE SOLUTION FOR TWENTY-FOUR PSYCHOLOGICAL TESTS 
STRUCTURE PATTERN 
Test 
Spatial 
Tin in, "jy, p Belkis Verbal Speed Memory 
i ^ ^ * * 
725 .403 -403 .461 -703 | —.068 .091 .035 
.162 .261 .210 .278 .462 | — 018 | — .009 .025 
564 .326 .195 .296 -595 .020 | —.081 | —.010 
-587 -378 .263 .316 -578 .067 | — 002 | —.051 
.449 .815 .419 .398 — .044 .827 .092 | —.057 
.454 .811 .336 .440 — .045 .828 | — 077 .086 
.440 .855 -371 .948 —.051 .931 .040 | —.125 
.552 .681 .457 .414 .219 .b16 -164 | —.076 
.452 .847 .274 .459 — .076 .903 | — 204 .161 
.133 .295 .718 .404 | —.316 .081 +179 078 
.278 .306 .T00 -538 | —.180 -066 .606 .245 
.345 .234 .715 .360 -138 | —.152 .794 | — 118 
.536 -412 -689 .392 -350 014 -645 | — 204 
.203 .338 .328 .584 | —.237 .176 .011 -575 
.258 .261 251 .927 — .068 .033 | —.097 .608 
.501 .310 .316 .567 .307 | —.080 | —.069 .471 
.236 .305 -379 .657 —.287 «047 .008 .T6b 
.434 .251 .460 .653 .145 | —.191 .128 .592 
.948 .296 .323 .492 .082 .021 .026 .418 
.995 .533 .339 .485 .973 -250 | —.050 sad. 
.538 .391 .543 -505 -321 | —.014 .324 .133 
.549 .535 .299 .541 .250 .279 | —.151 .339 
.693 .573 445 514 .489 .212 .079 .075 
.422 .528 .609 -539 | —.014 - 266 .387 .178 
Reduced Structure Reduced Pattern 
.823 .481 .977 .475 .824 .000 -001 | —.002 
.557 .951 441 .489 .001 .949 -005 | —.002 
.411 .415 .897 .938 — .001 .003 .899 | — 002 
.502 .446 -522 .869 —.000 | —.002 .002 .869 
* Based upon the centroid solution of Table 8.2. i 
TABLE 11.11 


TOTAL CONTRIBUTIONS OF FACTORS 


Grand total —11.381 
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the resulting reduced structure and pattern bring out the above relation- 
ships more clearly inasmuch as the points representing these composites lie 


exactly on the reference axes. 
In the factor patterns of Tables 11.6 and 11.10 the coefficients exceeding 


three-tenths have been printed in bold-face type to facilitate the naming of 
factors. It will be observed that in both of these patterns there are a large 
number of insignificant entries. From the descriptions of the variables, 
given in Appendix B.1, the names of the factors indicated in these tables 
are justified. 

It may be noted that in the reduced oblique factor patterns all the entries 
are zero, within errors of rounding, except those in the principal diagonal. 
This follows from the fact that the oblique reference axes are passed through 
the points representing the composite variables. Thus, the reduced uni- 
factor pattern from Table 11.10 may be written in the form 


94972 
8997s 
86974. 


Ue 
| " 
us = 


is = 82471 


(11.28) 


L] 


thirteen variables may be written in a 


The reduced oblique pattern for the 
hree of equations (11.28) to within 


similar manner and agrees with the first t 
five points in the last decimal place. 


The general stability of an oblique 
tion of the number of variables, can be observed from the patterns of Tables 


11.6 and 11.10. For the thirteen tests which are common to these two pat- 
terns, the corresponding factor weights are quite similar. The discrepancies 
between the insignificant coefficients are of lesser importance than those 
between the large weights. For the latter set the differences are all less 
than one-tenth except for the two values of b3i. The two patterns may then 
be considered as essentially the same for the common portions. This agree- 
ment may also be observed in the factor structures and is evidenced most 
clearly in the reduced structures and patterns. Hence the composite vari- 
ables may be taken as the best direct measures of the oblique factors de- 


termined in either solution. 


uni-factor solution, upon the reduc- 


2. SOLUTIONS BASED UPON INITIAL BI-FACTOR PATTERNS 


o compare oblique solutions for the same 
variables when based upon different initial patterns, the next two examples 
also will employ the psychological variables but will be based upon the 
bi-factor patterns of Tables 6.12 and 6.8. In transforming a bi-factor pat- 
tern to an oblique solution, the number of factors is frequently reduced by 


In order to make it possible t 


t 
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one. This is due to the fact that the bi-factor solution, in order to exhibit 
the great simplicity of form, sometimes must involve one more than the 
minimum number of factors possible for an equally adequate solution of 
some other type. The bi-factor pattern can be tested for its rank, say m, 
when it contains m or m + 1 factors. Then a reduced bi-factor pattern, 
involving m composite variables, can be calculated. The rank of this re- 
duced pattern is also equal to m. The composite variables may be con- 


TABLE 11.12 
REDUCED BI-FACTOR PATTERNS 


Fon Turrtzen PeycnoLocicAL Tests For Twenty-Four PevcHoLooicAL Tests 


Yee] Bo | B | Bs | BpQu) Vet | Bo | m | B | a B | B lou 


Wu....| .6268| .5332|......]......| .8229 .8417 
ur... .| 7328 6080|.. .... .9518| .9596 
us. -7286| .8989) .9099 
8551 
9137 
TABLE 11.13 
DIRECTION COSINES 
Fon THIRTEEN VARIABLES Fon TWENTY-FOUR VARIABLES 

An ^a ^a ^n ^n ^a ^n ^n 
-7617 | .7694 | .5857 | Bo..... -7564 | .7109 | .6597 | .7749 | 1.0000 

.6480 0 0 Bion. .6540 0 0 0 0 

0 .6388 0 ': | 0 . 7033 0 0 0 

0 0 .8105 | Bs..... 0 0 -7515 0 0 

Maias 0 0 0 .5143 0 

Biss 0 0 0 -3673 0 


sidered as m points in the original bi-factor space. It can be shown then 
that these m points are contained in a space of m dimensions which also 
contains the origin. Hence a transformation from the bi-factor coordinate 
system to a new oblique reference system of m dimensions can be ac- 
complished. 

The bi-factor patterns of Tables 6.12 and 6.8 can be shown to be (ap- 
proximately) of rank three and five, respectively. The composite variables 
(11.25) are employed, and the reduced patterns of Table 11.12 are obtained. 
The same notation is used throughout the parallel development, but the 
reader should have no difficulty distinguishing the two examples. Denoting 
the oblique reference axes by 6’s, their direction cosines with respect to the 
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bi-factor reference systems are given in Table 11.13. Then the correlations 
among the factors can be calculated and are presented in Table 11.14. 


TABLE 11.14 
INTERCORRELATIONS OF FACTORS 


Fon THIRTEEN VARIABLEB Fon TwENTY-FOUR VARIABLEB 


Bi Br Ba B Bs Bs Ba Bi 
JO! | save sea, es [erem nn Biss cx 1.000 |.......]---- 
586 | 1.000 |......- Bo. .538 | 1.000 |...... 
auae us y 446 .451 | 1.000 | 8... .499 .469 | 1.000 
Bas iv .586 .951 .911 : è 
[m .756 71 .660 715 


TABLE 11.15* 
OBLIQUE SOLUTION FOR THIRTEEN PsYCHOLOGICAL TESTS 


STRUCTURE PATTERN 
Test Couso- 
NALITY 
P " " Spatial Verbal Speed 
j TjB, "7B, "jB, fi B: Bs m 
.748 .472 .360 .T04 .049 .024 555 
.450 .261 .199 .453 — .005 — .001 .199 
.589 .284 .216 .654 — .082 — .039 .353 
558 854 .269 .D30 .036 .016 .313 
.498 .802 .383 .038 .171 .019 .644 
.460 .801 .354 —.012 .809 — .005 .640 
.440 .857 .339 — .084 .924 —.040 741 
547 .703 .421 .182 .bbb .090 .527 
.405 .849 .312 —.124 .950 — .062 737 
.186 .188 713 —.10 —.115 .814 .539 
.322 .325 -669 .021 .019 -651 449 
264 .266 -122 — .047 — .053 .T67 .527 
.491 .495 -725 .141 .142 . 598 .573 
Reduced Structure Reduced Pattern ` 
$ 
AB. n cra .823 .482 .367 .823 .000 .000 677 
u. Indes ROR .558 .952 .429 .001 .951 — .001 .904 
Me icc neni .401 .405 .899 .001 —.001 .899 .808 


a Based upon the bi-factor solution of Table 6.12. 


The complete oblique solution for the thirteen tests is given in Table 
11.15. Although based upon à bi-factor pattern involving four factors, there 
are only three oblique factors in this solution. The implication of this is that 
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a solution which fits the data about as well as the bi-factor pattern of Table 
6.12 can be made in terms of one fewer, but correlated, factors. Geometri- 
cally, the thirteen points representing the variables are approximated by a 
three-space in the oblique solution, whereas they are contained in a four- 
space in the bi-factor solution. Hence the communalities as given by the 
former solution are slightly less than those given by the latter. This may 
be noted from the last columns of Tables 11.15 and 6.12. The total contribu- 
tions of the three oblique factors are indicated in Table 11.16, where the 
grand total is also given. This number, of course, falls short of the total 
original bi-faetor communality (6.936), because the common-factor space 
is smaller. The difference (0.136) is so small, however, that the greater 


simplicity of having only three factors in the solution is well worth the 
sacrifice. 


TABLE 11.16 
TOTAL CONTRIBUTIONS OF FACTORS 
fi B: Ba 
Missae 1:501 [52 mesas ace 
DR — .049 C a one 
| V -000 | — .061| 2.049 


Grand total —6.800 


The oblique solution for the set of twenty- 
sented in Table 11.17. Here, again, there is one fewer of the oblique factors 
than bi-factors upon which the solution is based. The communalities repro- 
duced by the oblique solution are then slightly smaller than those from the 
bi-factor solution. These communalities are not calculated for the individual 
variables, but the total contributions of the factors are given in Table 11.18. 
The grand total of the contributions accounts for 96 per =: of the total 
bi-factor communality. 

As in the case of the two oblique solutions based upon the centroid pat- 
terns, there is excellent agreement for the thirteen variables common to the 
solutions of Tables 11.15 and 11.17. F urthermore, the oblique solutions of 
Tables 11.6 and 11.15 for the set of thirteen variables are in harmony; and 
this is true also for the two solutions of the set of twenty-four variables. 

_The oblique factors identified in any one of these solutions are practically 
identical with those of any of the others. Thus, there is evidence that an 
oblique solution is more or less invariant regardless of the particular form 
of the initial solution. What is important in order that the oblique solution 


shall be “almost unique,’ . 
of variables. : i. Wever, is that there be clearly defined subsets 


four psychological tests is pre- 


TABLE 11.17* 
OBLIQUE SOLUTION FOR TWENTY-FOUR PSYCHOLOGICAL TESTS 


À STRUCTURE PATTERN 
TrsT 
ria, | tia, | oe, | tum, | tas, | Spiel | Verbal | Speed | Memory DN 
j B Bs Bs Bi Bs 
.762 | .419 | .389 | .456 | .589 . 740) .001 .001| —.001 .028 
.456 | .254 | .236 | .277 | .357 .434 .001| —.001 .001 .028 
.617 | .285 | .265 | .311 | .401 .730|— .003| —.001; —.000| —.147 
.558 | .329 | .305 | .359 | .463 .484|— .001| —.002, —.002 .102 
.440 | .817 | .384 | .451 | .582 | —.001 .814| —.001| —.002 .006 
“435 | .802 | .379 | .446 | .575 | —.000| .795) —.002  .000| .011 
1404 | .878 | .352 | .414 | .534 | —.002| 1.006) — .003| —.002) —.175 
.472 | .707 | .412 | .484 | .624 .002 .b33| — .000 . 000! .243 
.424 | .840 | .369 | .434 | .560 .001/ .894| — .001 -003| — .079 
293 | .276 | .702 | .301 | .388 .002 .001 +793 .003| —.141 
394 | .370 | .703 | .404 | .521 .002|— .000 .636 .004 .096 
.306 | .287 | .749 | .313 | .404 .001|— .001 .853| —.001| —.158 
.436 | .409 | .709 | .446 | .576 .002|— .002 -582| — .000) .192 
.293 | .276 | .256 | .581 | .388 .001 . 000; -002 «701| —.158 
“965 | .250 | .232 | .517 | -351 | —.001|— .001 -002) .613| — .124 
375 | .353 | .327 | .566 | .496 -003} .002 .001 467) .136 
"319 | .300 | .278 | .694 | .422 .001|— .000 002! +920} —.294 
“390 | .366 | .340 | .571 | .515 -002}— .001 .001 .429 .181 
"334 | .314 | .292 | .445 | .442 .000|— .001| — -000 .257 .243 
"487 | .458 | .425 | .499 | .644 . 000 -000| -000 .000 .644 
“488 | .459 | .426 | .500 | .645 .001 . 000! . 000 .000 .644 
.487 | .458 | .425 | .499 .644 .000 .000 .000 .000 .644 
"555 | .522 | .484 | .569 | .734 —.001|— .001| —.003| —.002 .T40 
“539 | .506 | .470 | .552 | .712 — .000|— .002| —.001| —.003 .718 
Reduced Structure Reduced Pattern 
.842 ko .420 | .493 | .637 -842 -000 .000 .000 .000 
.516 | . .450 | .529 | .682 | —.001 .958 . 000. .002 .000 
“454 | .427 | .910 | .465 .600 .000|— .000 «909 -000 .000 
.501 | .471 | .437 | .855 .663 | — .002) .001| —.001 .851 .006 
“691 | .650 | .603 | .708 .914 | —.002|— .002| —.003) — .002 .921 
* Based upon the bi-factor solution of Table 6.8. 
TABLE 11.18 fg d Kan ^ 
5 TOTAL CONTRIBUTIONS OF FACTORS 
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11.5. Illustration of Inapplicability of an Oblique Solution 


Now an illustration will be given in which an oblique solution is not suit- 
able. Such a situation arises when there is no evidence of clearly defined 
clusters either by the method of B-coefficients or by inspection of a pre- 
liminary orthogonal pattern. The example of eight political variables of 7.5 
is employed for this purpose. The principal-factor pattern of Table 7.9 is 
reproduced in Table 11.19 for convenience. 

Although all the variables do not fall into distinct clusters, two subsets 
can be selected by inspection of the principal-factor pattern. It may be ob- 


TABLE 11.19 


INITIAL PRINCIPAL-FACTOR PATTERN AND FINAL OBLIQUE 
SOLUTION FOR EIGHT POLITICAL VARIABLES 


OnLtqvE SoLUTION 


PRINCIPAL- 
Factor PATTERN 
VARIABLE Structure Pattern 

Pi Pi "jx, "ix, ay TU 
€— MÀ .69 —.28 .74 SUBE 76 03 
M RRO .88 —.48 .99 —.69 1.16 22 
B. oidenori vn te .87 —.17 .87 —.78 68 | — .24 
M: sciente tas 9t —.88 —.09 —.79 .87 | — .29 64 
Bian t ue .28 65 .03 —.46| — .85 | —1.13 
6 89 01 .88| —.85 42| — .52 
Menai wa —.66 56 — .42 —.80 53 1.21 
[P —.96 —,15 —.84 96 | — .24 7T 


served that the groups (1, 2, 3) and (4, 7, 8) are in the fourth and third quad- 
rants, respectively. The eight variables are plotted in Figure 11.4, in which 
the grouping is verified geometrically. It is also evident from this figure 
that the clusters are not well defined in terms of the entire set of variables. 
Passing axes through the composite points determined by the above groups, 
the oblique factor solution is obtained by the procedure of 11.3 and is pre- 
sented in Table 11.19. 

The oblique factor pattern does n 
tion found in the precedin 
groups of variables, the fa. 


ot approach the uni-factor form of solu- 
E examples. Instead of being involved in distinct 
g i ctors are of the bipolar type. The interpretation 
is no simpler than in the case of the original principal-factor pattern and 
has the added complexity of correlated factors. Furthermore, the total di- 
rect contributions of the two oblique factors are 3.707 and 4.121, respective- 
ly, indicating level variances. Thus, while the essential principal-factor form 


e—a 
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is retained by the oblique solution, the important property of decreasing 
contributions of factors has been lost. Thus for the present example an 
oblique solution is evidently not desirable. 


=1.0 7.8 -6 -4 5.2 

e 4 
dee j 
-.4 


Fie. 11.4 


From the foregoing examples it is evident that the usefulness of the 
oblique form of solution depends upon the design of the set of variables. 


When the variables can be clearly grouped by the method of B-coefficients 
or from any preliminary orthogonal pattern, it is to be expected that an 
oblique solution of the uni-factor form can be obtained. Moreover, such & 
solution is independent of the particular initial pattern and is practically 
unique once distinct clusters of variables are determined. 


E ee Mi € 


PART IV 


ESTIMATION OF FACTORS AND RELATION- 
SHIPS BETWEEN FACTORS 


ET O R a pna —— 
: : 3 p — we o 


CHAPTER XII 
ESTIMATION OF FACTORS 


12.1. Introduction 

There are two basic problems with which factor analysis is concerned. 
The first of these deals with the methods for obtaining the linear resolution 
of a set of variables in terms of hypothetical factors. The preceding chapters 
were primarily devoted to the solution of this problem, the results being the 
preferred solutions. The second problem is concerned with the description 
of the factors in terms of the observed variables and is the subject matter of 
the present chapter. 

Since the total number of factors (both common and unique) exceeds the 
riables, the value of any particular factor for a given individual 


number of vai 
but can only be estimated from the 


cannot be obtained by direct solution 
observed values of the variables. The best prediction, in the least-square 


sense, is that obtained by the ordinary regression method. In 12.2 the linear 
regression of any factor on the n observed variables is obtained by the usual 
method. Thisis followed by an approximative method, which employs com- 
posite variables, in order to reduce the laborious task of the complete re- 
gression method. A method is presented in 12.4 which is superior to either 
of the preceding because it is more rapid and gives just as accurate results 
as the complete estimation method. This method, in effect, replaces the 
observed correlations of variables by those reproduced (or computed) from 


the factor pattern. 
Two other methods, 
the preceding ones, are also 


which at present do not seem to be as practical as 
given. In 12.5 a regression method is presented 
in which the sum of the squares of the unique factors is minimized. This 
method produces estimates of factors which are usually quite different from 
those given by any of the other methods. The final method for describing 
the factors in terms of the variables involves the mathematical solution of a 
set of equations rather than the statistical estimation by regression. Hence 
the factors themselves instead of estimates of them are obtained. Unfortu- 
nately, however, this solution is in terms of “ideal” variables (not the ob- 
served ones) and therefore cannot be employed in a practical situation. 

In order to simplify the development of the following sections, it will be 
convenient to formulate the essential concepts of the previous chapters in 
matrix notation. Let it be assumed that a set of variables has been ana- 
lyzed in terms of m common factors (Fi, Fa, +++ F,,) and n unique factors 
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(Ui, Us, ... , Un) as given by (2.16). Such a factor pattern may be written 
in the form of a matrix equation as follows: 
(12.1) Z-MF, 


where the column vectors 


Z= {zz haze et 
(12.2) c Pond Ui Wa: con 


represent the variables* and factors, respectively, and M is the complete 
pattern matrix, 


Gi Gi ... üm ay 0 wee D 
(12.3) M= ün Qn ... am O ae... 0 
Gnt Qs ... Onm O O ... ay 


If the matrix of coefficients of the common factors is represented by A and 
the diagonal matrix of unique-factor coefficients is denoted by U, then M 
may be written as the composite of these matrices, j.e., 


(12.4) M-|A Ul. 


It will be assumed that the unique factors are uncorrelated among them- 
selves and with all common factors, but, unless stated to the contrary, the 
common factors may be correlated. 

It was shown in 2.4 that when the common factors are uncorrelated the 
correlation of a variable with a factor is given by the corresponding coeffi- 
cient in the factor pattern. Hence, for the orthogonal case the factor struc- 
ture is identical with the matrix M. If the factors are correlated, however, 
this is no longer true. In that case the factor structure may be defined by 


tn he ciw f 0 O a « 0 
ta t caw bap 

(12.5) S=|T U|-|" ™ m O a ... 0 | 
la the $35 Ls 0 0 ... An 


where tj, = T;;r, and a; = Tap 

The notations here introduced will be used th: 
tional definitions will be made, as needed, in t 
of the various procedures. 


roughout the chapter. Addi- 
he course of the development 


* Strictly speaking, of course, the variables should be designated by primes on the z's. 
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12.2. Complete Estimation Method 
The rectilinear prediction of any common factor F,involves the determi- 
nation of the coefficients in the regression equation 


(12.6) FB, = Bazı + Bate ser Bat + + + © Bent 
(si 1,2, oo c ms 


ubseripts have been omitted from the con- 
on coefficients. A similar equation can be 
ue factors by employing it in place of F, in 
theory of multivariate regression, it can be 
s for the determination of the 6’s in (12.6) 


where the (n — 1) secondary s 
ventional notation for regressi 
written for any one of the uniq 
equation (12.6). By the general 
shown that the normal equation 
may be written as follows: 


Ba + nbat. + Tian = tie > 
(12.7) TuBa t+ Bat- + ronBen = im; 
Taba + rapat -et Ben = tne 
n be estimated when the correlations (t;s) of the vari- 


Thus any factor F, ca 
d correlations (7;,) among the vari- 


ables with this factor and the observe 
ables themselves are known. Of course, if a unique factor U; is being esti- 
mated, then all the correlations of this factor with the variables are zero, 
except for the correlation with 2; which is equal to a;. 

jons (12.7) can be simply obtained by the determi- 


The solution of equations 
nantal method (Cramer’s rule). To this end, define the matrix D of inter- 


correlations of variables bordered by their correlations with the factor Fh, 


as follows: 


Y du dm +++ i» 

te 1 ms Tin 

(12.8) D = || ta ra lil +++ Tn 
H 


ins Tni Tn2 


Then the regression coefficient of any variable z; in the estimation of F, is 


given by 

(12.9) Bee — pe 212..." 

where D., is the minor of the first element and D; is the cofactor of t, in D. 
ly the determinant E of observed correla- 


It should be noted that D. is mere 
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tions and that the determinants Dj, can be expressed in terms of the co- 
factors of the original correlation matrix R. Thus the values (12.9) may 
be written explicitly in the form 


(12.10) Bai = à lR + tose; +... + GBA, 


where Rs; is the cofactor of r,; in R. 
The regression equation (12.6) for F, may then be written as follows: 


(12.11) F, = t/R7Z, 


where t, is the column vector {tis tos... tne} taken from column s of the 
factor structure S. In particular, if F, is one of the unique factors, say U » 
this equation becomes 


(12.12) U;=|l0 0 ... a; ... O|R7Z, 


or, in expanded form, 
(12.13) Ü; = Run + Run +... + Ran]. 


If in equation (12.11), s takes on the values 1, 2,..., m, the results 
can be expressed in a matrix equation as follows: 


(12.14) f = TRZ, 


where f is the column vector {Fi Fz... Pa} of common-factor estimates. 
More generally, if the range of s in (12.11) extends over the unique factors 


as well as the common factors, then the matrix equation for the prediction 
of the entire set of factors becomes 


(12.15) F = YRZ, 


where F is the column vector of all the factor estimates. Furthermore, this 
equation represents the estimates for all the individuals if Z is regarded as 
the matrix of all the values of the set of variables. 

An entire set of factors, whether ortho 
by means of equation (12.15). When the 


pattern and structure coincide. Then 
the form 


(12.16) F = M'/R-Z (uncorrelated factors) . 
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If the factors are correlated, equation (12.16) obviously does not apply. 
Then the distinction between a pattern and structure must be clearly made. 

It is possible to write a formula for the prediction of correlated factors 
which explicitly employs the pattern matrix M. Let the matrix of correla- 
tions among the common factors be denoted by 


1 TRF, +++ TREES 
Tr,F, 1 TES 
(12.17) o= 
TRE, TFQFE, coc 1 


so that the matrix of correlations among all the factors is 
.|$ o 
(12.18) $- l2 I 


The relationship between a factor pattern and a structure is 


(12.19) S-M$, 


which is proved in Appendix B.11. Substituting this expression into equa- 


tion (12.15), the latter becomes 
(12.20) F = $M’/R-Z, 
upon recalling that $ is a symmetric matrix. This equ& 


(12.16) when all the factors are uncorrelated. mE 
Since the unique factors are of minor interest in factor analysis, it is con- 


venient to write a formula for the prediction of the common factors employ- 


ing the pattern matrix A. Substituting 


tion reduces to 


(12.21) T - A6, 


which relation is shown in Appendix B.11, into equation (12.14), this be- 


comes 
(12.22) Í-9$AR7Z. 
s are uncorrelated, this equation reduces to 


In case the common factor 
f = A'R?Z (uncorrelated factors) . 


(12.23) 


In the preceding formulas the pattern 
structure values. 


values are employed instead of the 
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A measure of the accuracy of estimation of a factor F, by means of equa- 
tion (12.6) is given by the coefficient of multiple correlation, which is desig- 
nated by R,. Several important and useful formulas involving R, will next 
be developed. The normal equations for the regression equation (12.6) may 
be written in the condensed form 


(12.24) ZF — Fade = 0 (Jo 3, 2,5. 470) 


where the summation extends over the N observations of each variable. 
It will be understood, in the following, that Z means “sum for i from i = 1 
to i = N,” although the 7 is omitted. Since the set of residuals (F, — F,) 
is orthogonal to each of the n sets of numbers z;, it is orthogonal to any 
linear combination of these z;. In particular, the set F, is such a linear com- 
bination, and hence 


(12.25) z[F, —F,JF, = 0. 
This may be written " - 
ZFF, = 2F?, 
which, upon dividing through by N, reduces to 
(12.26) OpTp p = cg? 


UE S 
F, F,F, F, 


The coefficient of multiple correlation of F, in terms of £i) iy. «wen 5718. 
defined to be the simple correlation coefficient of F, and F,. The expression 
(12.26) may finally be written in the form 


(12.27) Rr =o 


Fat 
This formula shows that the standard deviation of the factor estimates is 
equal to the coefficient of multiple correlation. 

A simple formula for the calculation of R, may 


be given now. Multiply 
both sides of (12.6) by F, and sum for the N ind 


ividuals, obtaining 
ZF,F, = Balak’, + BaZzF, +... + BuZz,P,. 


Upon dividing by N, this equation becomes 

Opre, P, 7 Bailie + Bala +... + Bau. 
Then, according to (12.27), 
(12.28) Ri = Batis + Bastar +... Bela. 


This is the simplest formula for the computation of the multiple correlation 
coefficient. 


Another formula for the calculation of R, can readily be obtained. Mul- 
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tiply the first of the normal equations (12.7) by Ba, the second by 2, etc. 
thus obtaining i í 
Batis = Ba. T Babati t- + Babanin» 


(12.29) Batos = [MES + Bis 4... Bafaston 5 


Bul = Babain + BarBaatna +- + oF Bis. 


Adding these equations, and employing (12.28), there results 


(12.30) JF = Ys T2 T BaibakT ik * 


j=l j<k=1 


This formula, although not so simple as (12.28) for computing R., illus- 
trates an important property. Any product term slija in (12.28) measures 
the total (direct and indirect) contribution of the corresponding variable X; 
to R2, or the importance of that variable as a “determiner” of F,. The reso- 
lution of the total contribution of any variable into its direct and indirect 
effect upon F, is indicated in (12.29). Thus, while Bah. represents the total 
portion of Ej which is due to Xi, the right-hand member of the first of 
equations (12.29) shows that this is composed of the direct, contribution 
(82,) of X; and of the indirect contribution (BaBsrtix) of Xi through its cor- 
relations with each of the other variables X+ (k = 2,3,...,7). It may 
be noted that the indirect or joint contribution of any two variables is 
distributed equally between them. 

While the determinantal method for the calculation of the regression 
coefficients, given by formula (12.9), has some undisputed theoretical ad- 
vantages, a more economical procedure is desired, especially when dealing 
with a large number of variables. Gauss’s method of substitution, which 
is described in Appendix G.1, isemployed for this purpose. The latter method 
produces a routine scheme for the evaluation of the 8’s and also furnishes & 
complete check on the arithmetical work. Two illustrations of the complete 
estimation of factors are presented in G.4. These illustrations are for the 
sets of principal factors and oblique factors for the eight physical variables. 

The values of the regression coefficients in the equations for the predic- 
tion of the oblique factors Yı (Lankiness) and y» (Stockiness) are given 1n 
Table G.5. The equations may be written explicitly as follows: 


yu = 2152; + 39120; + 20325; + -15824: + 04325; 
— .0092,; — -00527; — .00225; , 


(12.31) À 
Z. = — 0472; + 1362: — 07225; + -024za: + -615z5: 


yu = 
+ .1962;; + 07327; + .1632s: - 
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The values of these factors for a particular individual i are obtained by sub- 
stituting the appropriate standardized values in the above equations. The 
subscript 4 has been included in these equations to indicate clearly that 
values are substituted for the variables in order to get particular estimates 


TABLE 12.1 
VALUES OF EIGHT PHYSICAL VARIABLES FOR Two GIRLS 
Case 1 Case 2 
VARIABLE 
Xj ET Xj zja 
A 63.98 in. 0.01 66.34 in. 1.14 
2. 63.19 in. —0.42 66.89 in. 1.06 
3. 16.89 in. —0.31 17.99 in. 1.33 
4. 19.09 in. —0.62 20.71 in. 1.27 
5. 149.25 lb. 1.98 | 125.5 Ib. 0.41 
6. 13.15 in. 1.33 12.44 in. 0.26 
T7. 34.37 in. 1.65 32.52 in. 0.69 
8. 10.87 in. 1.42 10.55 in. 0.94 
TABLE 12.2 


MEANS AND STANDARD DEVIATIONS OF EIGHT 
PHYSICAL VARIABLES FOR 305 
FIFTEEN- YEAR-OLD GIRLS 


Variable Mean Sindara 
Deviation 
a M; ej 

1. 63.96 in. 2.09 in. 
2. 64.25 in. 2.50 in. 
3. 17.10 in. -67 in. 
4. 19.62 in. . 86 in. 
5. 119.22]1b. | 15:191b. 
6. 12.27 in. -66 in. 
T. Chest girth.......,.... Li. 31.21 in. 1.91 in. 
8. Chest width.............. 9.92 in. - 67 in. 


of the factors. In general, however, the secondary subscript is dropped for 
simplicity. 

The prediction of the above factors will now be illustrated for two in- 
dividuals whose measurements for 
12.1. The original observations Xj; are changed to Standardized values 
2; by application of formula (2.2). 
required for this change are given i 
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values z;; into equations (12.31), the resulting estimates become 


Estimates for Case 1: y, = —0.26, Ya = 1.78 ; 
Estimates for Case 2: y;, = — 1.21, Yo = 0.583. 


It can be seen from the eight standardized values for each girl in Table 12.1 
that the first is of the stocky type and the second is tall or lanky. While 
the original values reveal these facts, they can be indicated more simply by 
the above factor estimates. Although the estimated factors are not in stand- 
ard form, their standard deviations (.980 and .961) are nearly the same, and 
so the estimated values are approximately comparable. The estimates for 
the first girl indicate clearly that she is almost two standard deviations 
above the mean in the factor ‘“‘Stockiness”’ and slightly below the average 
in "Lankiness." The second girl is a less extreme type—being less lanky 
than the other girl is stocky and also being above average in “Stockiness.” 
In the foregoing illustrations standardized values were employed in the 
direct application of equations (12.31). The calculations of the standardized 
values for many variables for a large sample of individuals is laborious. The 
work can be greatly reduced by formally expressing the equations of esti- 
mation in terms of observed values by the use of formula (2.2). Thus, in 
general, an equation of the form (12.6) may be written as follows: 


Bay, ext... +B XL C, 
si 02 On 


(12.32) F, = 

where 

(12.33) o= bam p Papi +E Ma. 
[41 02 O5 


timated factor is not in standard form. Such 


a variable, however, has a mean of zero and a standard deviation which is 


equal to the coefficient of multiple correlation as shown by (12.27). For data 
which produce a high multiple correlation the estimated factor is almost in 


standard form. 
The values of the factors estimated by equation (12.6), or (12.32), in- 
clude both positive and negative numbers. If it is desired to eliminate the 
e to an arbitrary positive 


negative values, a transformation can be mad ary F 
dardizing the values Fs; given by 


scale. This can be accomplished by stan : i 
(12.32) and equating this variable to an arbitrary variable in standard 
form. Thusif the arbitrary variable Y is assigned a mean of 50 and standard 
deviation of 10, the required transformation can be written in the form 


10 = 
(12.34) y- = F,+ 50, 
F, 


It should be noted that an es 
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which reduces to 


(12.35) Y= E F, + 50 


upon substituting the multiple correlation coefficient for the standard de- 
viation of the estimated factor according to (12.27). Such transformations 
have been found especially useful in comparing factor estimates from a bat- 
tery of psychological tests.* 

An illustration of the proportions of the variance of one of the above esti- 
mated factors due to the eight physical variables will now be given. The 
direct and indirect contributions of these variables on the prediction of the 
factor “Lankiness” are indicated in Table 12.3. Each entry in the table 


TABLE 12.3 


PROPORTIONS OF VARIANCE OF COMPUTED yı DUE 
TO THE INDEPENDENT VARIABLES 


Total 
Contribution 
Bytj 


Variable 1 2 3 4 5 6 7 8 


“100 


.000| .000 
Indirect contri- Ri-.961 
bution...... .177, .216| .143| .116| .018|—.003|— 002 —.001 


BM 


proper represents the total indirect contribution ona of variables Xj 
and X,. The total indirect contribution of any variable is equal to one-half 
of the sum in the row and column representing that vari and is given 
in the last row of the table. The direct contributions (6};) are given in the 
row preceding the last. The total contribution (@,;t;:) of each variable is 
presented in the last column of the table. Of course, the sum of the direct 
and indirect contributions of each variable must be equal to its total con- 
tribution. Finally, the sum of the entries in the last column (or last two 
rows) is equal to the square of the coefficient of multiple correlation. 


* Karl J. Holzinger and Frances Swineford, A Study in Factor Analysis, 1939. 


ESTIMATION OF FACTORS 275 


Although the complete estimation method has been developed and illus- 
trated in the present section, it is not very practieal. The general formula 
(12.15), or (12.20), involves the caleulation of the inverse of an n X n 
matrix of correlations—a task which is very laborious as n becomes large. 
The present method, nevertheless, forms the basis for the simplified meth- 


ods of the succeeding sections. 


12.3. Approximative Method 

Several approximations for the estimation of common factors have been 
proposed* in order to reduce the matrix whose inverse is required. Such 
approximative methods involve the grouping of certain variables into com- 
posites. The simplest procedure is as follows: Combine the respective sub- 
sets of variables which best measure the factors and employ formula (12.14) 
in which all the symbols now stand for the corresponding reduced matrices. 
The approximative method is best adapted to the case of many variables 
which fall into a relatively small number of distinct subgroups. In applying 
formula (12.14) to such an example including, say, ™ subgroups, the m X m 
matrix of intercorrelations among the composite variables is used in the 
place of R. Then the major portion of the calculations is greatly reduced. 
The effect of this procedure is to give all variables of à subgroup equal 


weight. 

Em is desired to give varying weights to some of the individual variables, 
the preceding method can be modified slightly. For example, if the first 
four variables of a set are the best measures of the first common factor, they 
may be used individually, while all other variables are grouped into com- 
posites, for the estimation of this factor. The matrix whose inverse is re- 
quired then consists of the intercorrelations of the first four variables and 
the (m — 1) composite variables. When the second, and successive, factors 
are estimated, the corresponding variables which best measure them are 
retained, while all other variables are grouped into composites. 

To illustrate the approximative method, the data of the eight physical 
variables are again employed. Grouping the variables as indicated in (11.10) 

1 tions of (11.11), the composite variables may 


and using the standard devia 


be written as follows: 


Ll Cu SE 
(12.30) my ete tartan 
was 3417 i 


“Systems of Regression Equations for the Estimation of Fac- 


* Harry H. Harman, 
ional Psychology, 1938. 


tors,” Journal of Educat: 
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The correlation between these composites can be calculated by means of 
formula (2.42), yielding 


(12.37) Te =e = 4448, 


in which the correlations of the original variables are taken from Table 7.1. 
The estimation of the first oblique factor yı, for example, can be made from 
the regression equation 


(12.38) Yı = Bunu + fius. 
The normal equations (12.7) for this case become 


Bu + 444861: = .977 , 


(2.23) | 444881 + Bis = .455, 


where the correlations tn and tz of the composite variables with y; are taken 
from the reduced structure of Table 11.2. The solution of equations (12.39) 
are Bu = .9656, £i; = .0255, so that 


(12.40) Jı = .9656u + .0255u; . 


This equation cannot be used for the estimation of individual values of 
yı unless the values of the composite variables are computed. Rather than 
do this, however, equation (12.40) can be written explicitly in terms of 
the original variables by employing the definitions (12.36). Substituting 
the expressions for u; and wz in (12.40), this equation becomes 


(12.41) Yı = .258(21 + za + 23 + 24) + -007 (25 + zs + 2; + 23). 


This equation illustrates the fact that all variables of a subgroup are given 
equal weight by the approximative method. For the two girls, considered 
before, the values of y; given by (12.41) are 71 = —.30 and Ju = 1.25. 
The slight discrepancies between these values and those previously ob- 
tained are due to the grouping of the variables. 

The preceding illustration is given so that direct comparison, for the same 
data, can be made with the complete estimation method. One would not, 
ordinarily, use the approximative method when only eight variables are in- 
volved. The next illustration is more adapted to the present method. 

Consider the problem of estimating the common factors in the twenty- 
four variable problem of 6.10, the bi-factor pattern for which is given in 
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Table 6.8. The approximative method might be used to advantage for an 


example of this type. For purposes of estimating any of the factors B», Bi, 


Bo, Bs, the variables measuring Bs and B, may be grouped to form a single 
composite variable v, as indicated in (2.45), the effect being merely that 
variables 14 to 19 will be equally weighted. Of course, in estimating B4 or 
Bs, these variables may be given varying weights. Then to estimate the 
factor B, (Spatial Relations), for example, the following regression equation 


is employed: 

1242) Bi = paza + Bn + Bas Pun + Bats + Buts + Bu ta + Buts s 
where, for simplicity, & single subscript is placed on each £ to indicate the 
variable to which it is attached. The fs can be computed by means of a 
routine scheme such as that indicated in Table G.5. The necessary sta- 
tistics are the intercorrelations of the first four variables (in Table 2.2), the 
correlations of these with the four composite variables (in Table 2.8), the 
intercorrelations of the composites (in Table 2.4), and the correlations of 
the variables with the factor (in Table 6.8). The equation of estimation 


becomes 


B, = 468a 572, + -42623 + .18224 
da { — 114u: — .013us — 25u, — .371us . 


This equation can be written explicitly in terms of the twenty-four ob- 
dardized composite variables by the 


served variables by replacing the standardi: y t 
corresponding sum of original variables divided by the standard deviation 


of the sum, that is, 


(12.44) uo eb sen $6. 57 1,2,3,4,5)» 


Ov, Ov, 
where the groups of variables are defined in (2.35) and 


tions are given in (2.46). Then, 


the standard devia- 


+ .42623 + .18224 
— 027(z + % + 27 + 28 + 2) 


(12.45) — Q04(no zu + #2 + 25) 
— .032(zu + 215 + Zs + 217 + zis + £39) 


— 100(Z20 + 221 + Zo + 228 + za). 


B, = 468a + 15722 


Of course, if it is desired, this equation may be written in terms of the ob- 


served variables X; 
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12.4. Shortened Method 


Recently, a much-shortened method for the appraisal of factors has been 
developed.* This method, which is just as accurate as the complete estima- 
tion method, has the great advantage of replacing the nth order matrix of 
correlations by a matrix of order equal to the number of common factors. 
Since the number of such factors is relatively small compared to the number 
of variables, the labor of computing the reciprocal matrix is greatly reduced. 


The matrix of reproduced correlations, with ones in the diagonal, can be 
expressed as follows: 


(12.46) R = MM’. 


Substituting (12.4) for M and (12.18) for into this equation, it becomes 
o 1] fe] - me o1: fi 
ord tol = 14 vl 


| = , 2 
= | A$A' +T. 
This relation is useful in simplifying formula (12.22) for the estimation of 
the m common factors. Premultiplying both sides of (12.47) by A'U- there 
arises] 


(1247 R= |A U|- 


A'U-R = A'U-*(A$A' + U?) = (A'U-A$ + DA’, 


* This method was first presented, for the case of uncorrelated factors, by Ledermann, 
“On a Shortened Method of Estimation of Mental Factors by Regression," Psycho- 


metrika, 1939. It was then generalized by Harman, “On the Rectilinear Prediction of 
Oblique Factors," Psychometrika, 1941. 


T The proof of (12.46) can be made by the method indicated in Appen. B.11. The ma- 
trix of observed correlations is given by 


à R-LZZ, 


andif Z — MF is substituted into (i), the matrix of reproduced correlations, with ones in 
the diagonal, is obtained: 


g 1 W = lpp ' 
(ii) R = 5 MFFM - M(szP)w = mem’, 


where the last equality follows from (B.41). In replacing observed by computed correla- 
tions, the tacit assumption is made that the residuals vanish. To avoid additional sym- 
bolism, R is employed for both matrices, but it should be clear when it is computed from 
the observed variables and when it is computed from the factor pattern. 


i Throughout this and the following section it is tacitly assumed that none of the 
uniquenesses vanishes. For an excellent treatment of the con 


an trary case see Guttman, 
“Multiple Rectilinear Prediction and the Resolution into Components,” Psychometrika, 
1940, § 17. 


è 


Í 
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or, defining the following m X m matrix, 


(12.48) K = A'U2A$, 


this expression reduces to 


(12.49) (I+ K)A' = AUR. 


Now, premultiplying both members of this equation by (I + K)* and 
postmultiplying by R“, it becomes 


(12.50) AR = (I + K) AU". 


Then, substituting (12.50) for A'R^* in (12.22), there finally results 


(12.51) F = o + E)'AU7. 
Although this formula may appear to be more complex than (12.22), it is 
actually much simpler to apply. Aside from the inverse of the square of the 
diagonal matrix of unique-factor coefficients, which is obtainable without 
any effort, the only matrix whose inverse must be calculated is of order m. 
To show how the common factors can be estimated by the shortened 
method, employing the procedure of Appendix G, it is convenient to write 
formula (12.51) in another form. Premultiply both sides of this equation 


by [&(I + K) to get 


(12.52) (Ger E) = A'U~Z. 


It will be observed that the resulting matrix, on carrying through the mul- 
tiplications, on each side of (12.52) is of order m X 1. This matrix equation 
thus represents a system of m algebraic equations, obtained by setting the 
corresponding elements equal to eac ices in the right-hand 
member of (12.52) are quite simple, the left appears 
to be rather complex. This expression can be simplified, however, by sub- 
stituting the definition of K from (12.48), producing 


dump  aGéme-0rcauuee = (bci + A'UA). 
Finally, the system of m equations for the estimation of the common factors 


may be written in the form of the matrix equation 


Lf = AU-Z, 


t 


(12.54) 
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where 
(12.55) L=¢o"+4+J and J2AU-A. 


It can easily be shown that the premultiplier of f is a symmetric matrix,* so 
that the system of equations represented by (12.54) can be solved by the 
method outlined in Appendix G.1. 

In ease the common factors are uncorrelated, $ is the identity matrix, 
and formula (12.54) reduces to 


(12.56) (I+ J)f = A'U-Z (uncorrelated factors) . 


The recommended procedure in applying formula (12.56) to a numerical 
problem is as follows: Divide each element of the jth row of A by a}, for 
j =1toj = n; this gives the matrix U-*A, which occurs as a factor in J 
and the inverse of which also occurs as a factor in the right-hand member 
of (12.56). Then multiply A by U-?A, column by column,] which yields 
the m X m matrix J. Since J is a symmetric matrix, it is necessary to cal- 
culate only those elements which lie on and above (or below) the principal 
diagonal. Finally, add unity to each diagonal element of J to complete the 
determination of the m equations represented by (12.56). Then the solu- 
tion for the common factors can be carried out by the method of Appen- 
dix G.4. 

The procedure for estimating a set of correlated factors by means of 
(12.54) is quite similar to the preceding. After the matrix J is determined, 
it must be added to $^! instead of the identity matrix. The procedure for 
caleulating the reciprocal of a matrix is given in Appendix G.5. When the 
set of equations represented by (12.54) is determined, the solution for the 
oblique common faetors is obtained by the method of G.4. 

To illustrate the shortened method, a set of correlated factors will be 
estimated, and hence the most complex formula can be applied. Again, the 
example of eight physical variables will be used so that comparison with 
the solution by the complete estimation method can be made. In estimat- 
ing the factors y; and vy» of the solution given in Table 11.2, it is first neces- 
sary to have the uniquenesses of the eight variables. Since this oblique solu- 
tion is based upon the centroid pattern of Table 8.4, and since the commu- 


* To prove that L is a symmetric matrix it is sufficient to show that the transpose of 
this matrix is equal to the matrix itself, namely, 


L' = ($7! + A'U-*A)’ = ($=) + (A)'(U-?)' (A)! = $-! + A'U2A =L, 
since $ (and hence $-!) is symmetric, as is also the diagonal matrix U-?. 


T This is the same as the conventional row-by-column multiplication of A’ by U-?4. 


nalities remain ir 
tracting the calculated communalities 
of coefficients of the common factors and the matrix of uniqueness 


.894 
.956 
| .932 
.879 
.005 
—.025 
—.060 
.080 


Dividing each ele 
sponding row of Us 
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.051 
—.027 
—.052 

.029 

.930 

.825 

.769 

685 | 


U-A 


v: = 


.154 


5.805 
8.613 
5.326 
4.351 
.038 
—.074 
—.133 
.170 


ll 
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nvariant, the uniquenesses can be obtained simply by sub- 
of Table 8.4 from unity. The matrix 


es follow: 


175 
.202 
.132 
.339 
.450 
471 


ment of the jth row of A by the element in the corre- 
the following matrix is obtained: 


331| 
—.248 
—.297 

144 
7.045 
2.434 
L0. 
1.454 | 


column-by-column multipli- 


alculated upon the 


The matrix J can then be c 
namely, 


cation of A by this matrix, 
22.236 —.099 | 


jede pome | — 099 10.913 


(12.57) 
For the present example the matrix of correlations of the common factors is 


1.000 -484 | 
$ =|| 484 1.000] 
and the inverse of this matrix, calculated in Appendix G.5,is 
a 1.306 —.632 | 
ge | —.632  1305|' 


(12.58) 
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The sum of the two matrices in (12.57) and (12.58) is given by 


E _ |23.542 —.731 
(12.59) ba ta a Ne | —.731 tamis |- 


Completing the indicated multiplications in (12.54), this equation becomes 


23.54271 — 73172 
—.7317ı + 12.21872 


..|[5.8052, + 8.6132; +... + .1702, 
© i] -331a — .24325 +... + 1.4542, 


or, upon equating the corresponding elements, the following pair of algebraic 
equations arise for the solution of 7 and 72: 


12.60 23.542y, — — .78ly; = (5.8052, + 8.6132 +... + 17023), 
(12.60) —.731yı + 12.21872 = ( .331lzı — 243z: +... + 1.45424) . 


The set of equations (12.60) can be solved by the method of Appendix G, 
as indicated in Table 12.4. Then the equations for the estimation of y; and 


TABLE 12.4 
SOLUTION OF (12.60) BY GAUSS'S METHOD OF SUBSTITUTION 


COEFFICIENTS OF 


T CIEN VARIABLE 
CHECK 
n y 1 | 2 | 3 | 4 5 6 7 8 
Forward Solution 

23.542 | — .731 5.805| 8.613| 5.326| 4,351 .038|— .074|— .133 . . 
- 1l. .081 |— .247)— .360|— .226|— .185|— .002 003) n - bor -41:905 

.331|— .243|— .297| -144| 7.045| 2.434 1.709| 1. . 
.180| .207| .165| .135| -001|— .002|— .00. “068 A 
.511 .024|— .132| -279| 7.046| 2.432| 1.705 1.459 25.510 
— .042|— .002 .011|— .023|— :578|— :199|— .140|— .120 | — 2:093 

Back Solution 

Bil eas .042 .002|— .011 -023 -578| -199 -140 SAT enaa 
Bij... -248| .366/  .226| .184|  :020|  :003|— OZ AIT. [Eee 


*» may be written in full as follows: 


Yı = .2482; + .3662; + .2262; + .1842, + .0202; 


+ .0034, — .0022; + .0112,, 
(12.61) ¢_ 
Y2 = 0422, + 00222 — 0112; + 0232, + .5782, 


+ 1992, + .1402, + 1202, . 
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These equations agree very well with (12.31), the minor discrepancies being 
due to the fact that the correlations computed from the factor pàttern are 


not exaetly equal to the corresponding observed correlations. The coeffi- 


cients of multiple correlation, Ri = .978 and R: = .958, indicate that the 


equations (12.61) for the prediction of yı and v; are just as reliable as the 


equations (12.31), for which E; = .980 and R: = .961. For the two girls 


whose measurements on the eight physical variables are given in Table 12.1 
the values of the factors, as estimated by (12.61), are 
yu = —0.28, Yu = 1.80, 


Fn = 122, Yor = 0.56, 


which are practically identical with the values previously obtained by the 
use of (12.31). 


12.5. Regression Method Employing an Alternative Principle 

Instead of the ordinary regression method for estimating factors, another 
method has been proposed by Bartlett.* Whereas in the previous methods 
the sum of the squares of the discrepancies between the true and estimated 


factors over the range of individuals is minimized, in the procedure now to 
f the unique factors over the range of 


be described the sum of the squares o: 
variables is minimized. This method is in harmony with Bartlett’s principle 
that unique factors should be introduced only in order to explain discrep- 
ancies between observed values and postulated general or group factors. 
According to Thomson, “Bartlett’s estimates and the regression estimates 
attain different ends, and it is agreed that each method is correct in the 
right place.” t 

Suppose there are just 
may be written as follows: 
(12.62) z; = anFı + ajFs + a;Ui G= L Benat 


The explicit expression for the unique factor of any V 


two common factors, so that the factor pattern 


ariable z; is 


U; = z (z; — agi pe ajF?) , 


(12.63) 
and the sum of the squares of all such factors may be denoted by 
as E (s; — anh — al). 


j= 


1264) F(R, F) = >, U} 
j=l 


* M. S. Bartlett, “The Statistical Conceptio: 


Psychology, 1937. 
1 G. H. Thomson, 


n of Mental Factors," British Journal of 


“Methods of Estimating Mental Factors," Nature, 1938, p. 246. 
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Then to minimize the sum of the squares of the unique factors over the 
range of variables, it is necessary that the partial derivatives of the func- 
tion F with respect to F, and F» vanish, i.e., 


oF 1 

am" 2z E: (z; — aay — ajsF;)ai = 0, 
(12.65) ! 

oF 1 

oF > 22 a (z; — aaFy — aj2F2)aj2 = 0, 


where the summations extend from j = 1 to j = n. These equations may 
be put in the form 


OhY pi anai) p, _ an, 
(z$)n*(xt)n-x4. 


(12.66) ý 


QjoQj " a3, — ap, 
(X) ne(X2)R-XE 


where primes have been placed on the F's to distinguish these estimates of 
the factors from the true factors. These equations involve the two un- 
knowns, Fi and F;, which may be determined in terms of the a’s and z's, 
as follows: 


(12.67) F; = M og (212, 


j=1 


where the c's can be evaluated from equations (12.66) by Gauss's method 
of substitution. 


A set of equations of the type (12.66) can be written, more generally, in 
matrix form, as follows: 


(12.68) JÉ-AU-z, 


in which the matrix J is defined by (12.55). Formula (12.68) can be applied 
to problems involving any number of common factors if it is desired to esti- 
mate them according to the principle of minimizing the unique factors. Al- 
though formula (12.68) is not a special case of formula (12.54), it may be 
noted that, if the term $-! is dropped from the matrix L in (12.54), that 
formula becomes identical with (12.68). The computations required in the 
application of formula (12.68) are the same as that described for the short- 
ened method, except that nothing is added to the matrix Je 
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The estimates of the common factors by means of formula (12.68) are 
ernative solution to any of the regression estimates of the pre- 
artlett pointed out that the principle of estimation adopt- 
letely agree with the solution that has usual- 
ifference does not affect the relative 
weights assigned to the variables in estimating à single generalfactor. When 
factors are involved, however, the discrepancy between 
ven more serious. Bartlett states: “One 
point of view appears to have been to consider all the persons with different 
possible factorial make-ups that would give rise to the observed test scores 
of a particular person, whereas I have regarded the test scores as a sample 
of all the possible scores that might have arisen for that person according to 
the different values of specific [unique] factors he may happen to have."* 


The present method will be applied to the estimation of the two oblique 
which have already been predicted by means of formulas 
The matrix J for this problem is given in (12.57), and 
ix (A/U-?) is given immediately above it. Then 
) can be solved by the method of 


then an alt 
ceding sections. B 
ed in this section does not comp 
ly been employed, although the d 


several common 
equations (12.54) and (12.68) is e 


factors, yı and Yz 
(12.14) and (12.54). 
the transpose of the matr 
the set of equations represented by (12.68 


Appendix G, producing 


/2 261a + 3872 + 24025 + .1962; + -00525 


i 
i — .0022, — .0052; + .00825 , 
(12.69) af = 083a — 0192. — 0252s 4.0152, + 6462; 

4.22325 + 1972; + 19325 . 


For the two girls previously considered the factor estimates by (12.69) are 
as follows: 
yh 1 


= —0.35, yh = 2.03 ; 
yi = 2 


1.28, ig = 0.56. 


on by means of equations (12.69) can be judged 
d error or multiple correlation coefficient. It 


formula (12.28) does not yield the required 
ee » noted, hoeven sent ibi. If formula (12.28) is applied, 
anata dm correlation” will be found to be equal to esi This 
follows from the fact that the common factors are i aen oe t pros 
dition that the unique factors are minimized, and the uniqueness 
standard error of estimate in a pattern equation. 


The reliability of predieti 
by the appropriate standar 


* Op. cit. 
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For the method discussed in this section the coefficient of multiple corre- 
lation is given byt 


Ju 
(12.70) =l- 


where J,, is the minor of the element in row and column s of J. In general, 


it can be shownf that the multiple correlation coefficient can be computed 
from the formula 


(12.71) RP=1- Lu 


where L is defined in (12.55) and L,, is the minor of the element in row and 
column s of L. 


The multiple correlations for the estimates (12.69) as given by (12.70) are 


Rı = V1 — 10.913/242.652 = 977, 
R: = V1 — 22.236/242.652 = .953 . 


These values are just about equal to those obtained for the estimates 
(12.31) and (12.61). It may then be concluded that, statistically, the dif- 
ferent methods for estimating factors are equally good. The choice of the 
method of this section instead of one of the other (equivalent) methods must 
be made on the basis of the principle of prediction which is involved. 


12.6. Factors Expressed in Terms of Ideal Variables 


Before leaving the subject of estimation of factors, 
will be presented. The present method is designed only for the description 
of the common factors—the expressions for these factors being in terms of 
hypothetical rather than the observed variables, and so are not immediately 
applicable in practice. 

The common-factor portion of a pattern is r 
in matrix form as follows: 


(12.72) Z* = Af, 


where f is the column vector {F; F, . 
the column Vector {z/’ zi’ . 


one more approach 


equired and may be written 


> T Fs] of common factors, and Z* is 
- + £y ] of the statistical variables projected into 


1 See Bartlett, ibid. 


1 For the orthogonal case see Guttman, op. cit. $10, and D £e i 
1 : à j wyer, "The Evaluation 
of Multiple and Partial Correlation Coefficients from the Factorial Matrix,” 
metrika, 1940, p. 229. Btrix," Psycho- 


` The inverse matrix ma 
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the common-factor space. To solve for i 
: 1 b the factors in terms of the hypo- 
thetical variables (z/^), proceed as follows: Premultiply both sides of (12.73) 


by A’, obtaining 


(12.73) N'Z* = A'M. 
Let 
(12.74) A^- AA, 


ymmetrie matrix of order m. Finally, pre- 


which is readily seen to be a s 
A“ to obtain the result 


multiply both sides of (12.73) by 
f = AA'Z*. 


near descriptions of the factors in terms 


(12.75) 


This matrix equation gives the li 
of the variables z/^ which are involved in the matrix Z*. 

To illustrate the present method, formula (12.75) will be applied to ihe 
example of eight physical variables. The matrix A of coefficients of the com- 
mon factors yı and 72 is given on page 281. The matrix A, calculated ac- 
cording to its definition (12.74), and the inverse of this matrix follow: 

3.365 —.011 AoE | .297 .001 | 

= | —D11 2.613 | ; = || 001 .383]' 
y be computed by the method outlined in Appen- 
Je case of a second-order matrix, directly according 
rse in Appendix A. The only calculation remain- 


dix G.5, or, for the simp 
A^ by A’. Then the expressions for the factors 


to the definition of an inve 

ing is the multiplication of 
can be put in the form 

a, = 2062 + 2844 + 27 Tet! + 26124’ + 002z; 

— 0072/" — .0172/ + .02425' , 


eal, ya = 0202 — 00947 — 01927 + 01224’ + 3562; 

+ .3162' + 2942!’ + .26225' . 
s of the two oblique factors (not 
factor portions” of the original 
are not known, and hence (12.76) 


i ues 0 A 
variables, The vel values of the factors for the indi- 
er—replacing each zj’ by Zi. 


cannot be applied direc : rey pma 
viduals. An approximation can e made, howe 
When this is done, the estimate may be denoted by double 


2.76) give the description 
s of the "common- 
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primes. Thus, the estimates of y; and y; by (12.76), employing the values 
of the eight physical variables given in Table 12.1, become 


Estimates for Case 1: y// = —.036, vai = 1.98; 
Estimates for Case 2: yj = 1.31, Yz = 0.08. 


The method of the present section can be applied in another situation 
with less misgiving than that involved in the foregoing example. When a 
factor analysis is made under the assumed composition of variables into 
common factors only,f then the investigator is justified in replacing the Z* 
matrix by the Z matrix throughout this section. 


T This assumption implies the following linear expression for the approximation of any 
variable 2; instead of the expression (2.4): 


2j = aifi + 4j2F2 +... 4+ ais P G =1,2,...,2), 
where m < n and the prime is used to distinguish the theoretical from the observed vari- 


able. Under this assumption the analysis is carried to the point where there are no more 
statistically significant (common) factors. 


CHAPTER XIII 


RELATIONSHIPS BETWEEN DIFFERENT 
FACTOR SOLUTIONS 


13.1. Introduction 


The present chapter is designed to show the essential unity of factor 


analysis. Although the preferred solutions which yield the linear expres- 
sions of the variables in terms of factors have certain distinct character- 
istics, they can be transformed from one into another. The problem of relat- 
ing two sets of factors, obtained by different methods of analysis, was first 
considered by the authors in 1937,* but the present treatment is somewhat 
more general. 

In 13.2 the general procedure for transforming one solution into another 
is presented. This is followed in 13.3 by a numerical illustration, employing 
the bi-factor and oblique solutions of the set of thirteen psychological vari- 
ables. Finally, in 13.4 the complete plan for obtaining a factor solution is 
outlined and should serve as a summary of the methods of factor analysis. 


13.2. Relationships among Factors 
In mathematics the reference syst 
lar configuration of points is of prime impo 


is of much lesser significance. Thus, if it is esired to 
get an algebraic equation for the ellipse, it is quite irrelevant whether rec- 


tangular Cartesian coordinates, nonrectangular Cartesian coordinates, or 
polar coordinates are employed. Furthermore, the particular orientation 


of axes isimmaterial. With each change of the coordinate system, of course, 
e will generally change, but the fact remains that 


the equation of the ellips ; 
the equation in each case describes the ellipse with respect to the given ref- 
erence system. 

The object of factor an 
propriate frame of reference, 


variables being of lesser signifi I 
problem is obvious. In selecting 2 particula: 


tors along the coordinate axes represent the fae 


system can be rotated about its origin in anin ape 
mon-factor space, there arises an infinite number 0 a 


given body of data. 
n, 


* Holzinger and Harman, ; 
Analyses," Journal of Educational Psychology, 
289 


em plays a very minor role; the particu- 
rtance, and the coordinate system 
desired to describe an ellipse, i.e., 


lysis, on the other hand, is the selection of an ap- 
the configuration of points representing the 
ance. Then the indeterminacy of the factor 
r reference system, the unit vec- 
tors, and, since the reference 
finitude of ways in the com- 
ctor systems for à 


between Factors Obtained from Certain 


“Relationships 
1937, pp. 321-46. 
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The problem of relating two factorial solutions involves the determina- 
tion of the relationships among the coordinates of the two systems. This 
may be accomplished by finding a matrix of transformation which carries 
the coordinates of one factor pattern into another. Thus, if the first factor 
pattern is denoted by A and the second factor pattern by B, then the prob- 
lem is to find a matrix T such that 


(13.1) AT=B. 


The matrix A represents the coordinates ja of the n points with respect to 
one set of m common-factor axes, say Fy, Fz, . . . , Fm; the matrix B repre- 
sents the coordinates b;, of the points with respect to a new set of axes, say 
Kı, K», . . . , Km; while the matrix T represents the transformation of the 
coordinates in A to those in B. The foregoing transformation implies that 
the number of axes in the two reference systems is the same. Actually, how- 
ever, it is only required that the n points be contained in the same space of m 
dimensions, although the second reference system might employ a larger 
number of coordinate axes. 

If the number of factors is equal to the number of variables then the solu- 
tion for T is simply 


(13.2) T=A"B. 


Since the number of factors is usually much smaller than the number of 
variables, however, the matrix A does not have an inverse and T cannot be 
calculated directly. It may be noted that, for any matrix A, 


(13.3) (AA) (A'A) = I, 


and hence, if both members of (13.1) are premultiplied by (A'A)7!A', there 
results 


(13.4) T = (A'A)A'B. 


This formula gives the desired matrix of transformation. 

The distinction between the problem treated here and the transforma- 
tions considered in Chapter X should be noted. In relating two solutions, 
the factor patterns (A and B) are assumed to be known, and the matrix of 
transformation (T) from one to the other must be determined. In obtaining 
a derived solution, as in Chapter X, only the initial pattern (A) is given, 


and a matrix of transformation (T) is built up to produce the desired final 
solution (B). 


The relationships among the 


factors themselves may also be obtained by 
means of the matrix T. Witho 


ut loss of generality the column vectors of 


— ás (A Qa = 


WI m Áo 
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the first and second sets of factors may be taken to be F — (F4 F: Fs} and 
K = (K; K: Ks Ki], respectively. From the definitions of the two factor 


patterns, and their assumed equality, it follows that 


(13.5) AF = BK. 


Premultiplying both sides of this equation by (A'A)7!A', and again employ- 
ing (13.3), there results 
(13.6) 

By making use of (13.4), this 
(13.7) 


This is the matrix formulation o 
Fo, F; and Ki, Ka, Ks, Ks. 1 

The important distinction between the relationships (13.1) and (13.7) can 
be exhibited clearly for the case of three factors in the first solution and four 
factors in the second. The matrix T may be represented by 
a d à Rh 
Co dz e f. 2 
ca ds es fs 
s calculated by means of formula 


four algebraic equations can be 
es of the second solution in terms 


F = (A’A)1A’BK . 
expression finally simplifies to 
F-TK. 


f the relationships between the factors Fi, 


T= 


(13.8) 


and f's are constant 
x equation (13.1), 
of the coordinat 


in which the c's, d's, e's, 
(13.4). From the matri 
written for the expression 
of those of the first, as follows: 

ciaj + cas F Catia 

diaj + d:a; + dijs 
(18.9) eaj + exiis + estis = bis, 

fiain + ftiz + frais = bj, 
(13.7), three algebraic equations for the 
f the second set can be written 


= ba, 


= be 
Bi (Poder 


matrix equation 


Similarly, from the 
t set of factors 1n terms 0 


expression of the firs 


as follows: i 
Fi= Ki + d,Ka + aks +fiks, 
= Ky + foKs, 

13.10 Fa = Ki t+ d;Ks + es 
| [nro set oe TK. 


rix of coefficients of (13.9) 


ranspose of the mat 
tem are coordi- 


that the t z 
It may be noted r, the variables in the former sys 


appears in (13.10). Howeve 
nates, while those in the latter are factors. 
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A system of equations of the type (13.9) gives the actual transformation 
of coordinates between the two factorial solutions. Thus the factor weights 
forany variable in one solution are expressed linearly in terms of the weights 
of the other solution. Equations (13.9) may also be used as a check on the 
calculation of the elements of T since the factor weights of both solutions 
are known. 

An alternative way of expressing the relationship between two factorial 
solutions is afforded by a system of equations of the type (13.10). Since all 
factors are assumed to be in standard form, the variances of the F's can 
be simply calculated from (13.10) in terms of the variances of the K's. For 
example, if the K's are uncorrelated, 100c? is the per cent contribution of 
the factor K; to the variance of F;. Equations (13.10) might also be useful 
in estimating the factors F if the equations of estimation of the K's are 
known. 


13.3. Numerical Illustration 


A detailed numerical illustration of the procedures of the last section will 
now be given, employing the example of thirteen psychological tests. The 
relationships between the bi-factor solution of Table 6.12 and the oblique 
solution of Table 11.6 will be obtained. The matrix of coefficients of the bi- 
factor pattern will be denoted by B, while that of the oblique factor pattern 
will be denoted by A, in agreement with the general treatment of 13.2. For 
the convenience of the reader these matrices are repeated here: 


Yı Y2 ys By B; B: B; 
-731 —.089  .142 .614 .425 
.441  .004  .004 -339 .296 
.721 —.090 —.142 369 .475 
508 .090 —.003 .460 .320 
—.058  .801 .087 .654 .467 
.037  .809 —.051 .604 .526 
A =|| —.068 .901 —.030 " B = || 578 645 
155  .591  .078 718 .235 
—.068  .919 —.081 .532 .689 
—.385  .164  .809 .244 .703 
—.039  .077  .659 .423 .520 
.073 —.177  .778 .346 .641 
.351 —.061  .594 .644 .429 


—_—_—_—_— € O A "A A mE 
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The first problem is the determination of the matrix T which transforms 
A into B. This is given by formula (13.4). In applying this formula, the 
product A'A is first caleulated. This may be done most simply by computing 
the sums of the squares of each column of A and the sums of the paired 


products of the columns two at a time, giving 
1.823 — .232 — .058 


A'A = || = .232 3394 — 017]. 
— 058 — .017 — 2.103) 


The next step involves the calculation of the inverse of the preceding 
matrix. The outline for the calculation of an inverse is given in Appendix 


TABLE 13.1 
FORWARD SOLUTION FOR CALCULATION. OF (A'A)7! 


v Identity Matrix 
— .058 1 0 0 
032 —.549 0 0 
— .017 1 0 
— .007 127 0 0 
— .024 127 1 0 
007 —.038 —.297 0 
2.103 0 0 1 
— .002 .032 0 0 
— .000 .001 .007 0 
2.101 .033 .007 1 
ils —.016 —.003 —.476 


Is for the present case are exhibited in Table 
btained by means of the back solution, em- 


6, and 2, as follows: 


G.5, and the numerical detai 
13.1. The inverse matrix is o 
ploying the values in lines 11, 


.554 .038 .016 
(A'À)7 = .038 .297 .003|. 
.016 .003 .476 
tted in Table 13.1 because of the check afforded 


The check column drin atrix by its inverse being the identity 


by the property of the pro 
matrix, namely, 


duct of a m 


1.000 .000 -001 
.000 .999 — .002 
.001 —.001 1.000 


(A'A)AÀ'A^ = 
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After obtaining the inverse matrix it is postmultiplied by A, row by 
row,* which yields 


(AAA! = 
404 .245  .394.285 —.000 .050 —.004 .110 —.004 —.194 —.008  .046  .202 


002 .018 .001 .046 .2386 .242  .265.182 .270 037 .023 —.047 —.003 
079.009 —.056 .007 .043 —.021 —.013 .041 —.037 379 313 369 288 


Finally, upon postmultiplying this matrix by B, the desired matrix of trans- 
formation is obtained: 

.808 .523  .047 —.024 
457 .021  .637  .007 
.590 .012 —.015  .789 


T = (A'A)7A'B = 


The actual rotation from the oblique coordinates to the bi-factor coordi- 
nates is given by 


bio = 808a; + .757a5, + -590a;3 , 

ba =  .528ag + .021aj, + .012a;; 
13.11 n 
( ) bz = —.047aj + .637aj, — .015aj , 

djs = — .024a; + .007aj; + -789a;3 , 


where the a’s and b’s are the coordinates in t 


he matrices A and B, respec- 
tively. It is evident from these e 


i quations that the coefficients of the first 
bi-factor (general) are composed of fairly uniform portions of the corre- 


sponding coefficients of all three oblique factors. The coefficients of each 


of the remaining bi-factors are composed almost entirely of fractional parts 
of a single oblique factor coefficient. 


Similarly, by means of the matrix T, the relationships among the factors 
may be exhibited as follows: 
^n = .808B, + .523B, + .047B, — .024B, , 
(13.12) Y2 = .757By + .021B, + :637B, + .007B, Y 
Ys = .590B, + .012B, — .015B + -789B;. 


From these equations it appears that 
of a substantial portion of the 
factor of the bi-factor solutio 


each of the oblique factors consists 
general factor and the corresponding group 
n. For example, the factors By and B; con- 


* This is equivalent to the conventional row-by-column multiplication (A’A)- by A’. 
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spectively, to the unit variance of ys. 


tribute 57 per cent and 41 per cent, re 
uares of the coefficients in the respec- 


It may be noted that the sum of the sq 
tive equations of (13.12) is not exactly unity. This discrepancy is due to 


the fact that the assumption of the equality of the two factor patterns is 
That is to say, the communalities and the correlations 
are not quite the same; or, in geometric 


do not define the same points precisely. 


met only roughly. 
reproduced by the two solutions 
terms, the two sets of coordinates 


13.4. Suggestions for Order of Analysis 
: From the various methods of analysis presented throughout the text it 
is evident that the choice of à particular form of solution is somewhat arbi- 
trary. In all cases, however, 8 certain routine of analysis must precede the 


final choice of solution. An effective outline which may be followed for all 
problems is given below. The first seven steps deal.with the preliminaries, 
while the eighth step for any problem is to select a particular form of the 
final solution. Following the outline there will be a brief recapitulation of 
the bases for the choice of a preferred solution. 


. Formulate the problem. 
Decide upon a set of variable 
. Caleulate all the product-moment c 


s relevant to this problem. 
orrelation coefficients. For most rigorous 


results the distributions of the variables should be investigated for normality 
and the correlations tested for linearity. Reliability coefficients may also be 
helpful in determining the adequacy of the variables as effective instruments of 


measurement. 
4. By inspection, rearrange 
lated variables. e 
5. If each variable in a subset has high positive 
elations wit 


and negative corr h 
easurement for each of these varia 


w yve 


the variables into tentative subsets of highly interre- 


correlations with all other variables 
h the remaining variables, reverse 
bles and thus obtain positive 


6”. If, after the reflection of variables, 


in this group, 
6/. If all the correlations 


positive, the metho 
cients can be employed as an objec- 
determining dis- 
jables. The num- 
mines the 


tive procedure for 
tinet subsets of var 
ber of groups roughly determi 
rank of the correlation matrix. 


there are still negative correlations, 
then the method of B-coefficients 
does not apply, and an approxima- 
tion to the rank of the correlation 
matrix is not obtainable at this 
stage. Similarly, when a matrix eon- 
tains positive correlations through- 
out but does not indicate distinct 
clusters of variables, then the rank 
cannot be approximated by the 
method of B-coefficients. 
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7'. Calculate complete estimates of the 7”. When complete estimates of the 


communalities of the variables, as communalities are not available, 
described in 7.2, by the direct meth- some arbitrary choice of the com- 
ods when the rank is one or two, or munalities must be made. One of the 
by the bi-factor method for higher simplest methods is to select the 
rank. Then obtain a centroid solu- average of all the correlations for 
tion based upon these communali- each variable as its communality. 
ties. 'This gives rise to the averoid solu- 


tion described in 8.4. 

8a. Calculate a bi-factor solution as the final form by the method described in Ap- 
pendix C. 

8b. Calculate a principal-factor solution as the final form. For a small number of 
variables this may be obtained directly, when the communalities are known, by 
the method of Appendix D. Otherwise, the principal-factor solution can be ob- 
tained by transformation from an initial solution as described in Chapter IX. 

8c. Caleulate a multiple-factor solution as the final form by the methods described 
in Chapter X. 


8d. Caleulate an oblique solution as the final form by the methods described in 
Chapter XI. 


When the analyst has reached Step 8 in the foregoing outline, he will 
need to consider the bases for choice of one of the preferred forms of solu- 
tion. As indicated throughout the book—especially in Chapters I, V, and 
XI—such a choice depends upon certain statistical criteria as well as upon 
the nature of the problem and the data. If the investigator starts with a 
theory in a given field, such as Spearman's theory of intelligence, then he 
might select the particular form of factor solution which would verify the 
theory. In another type of problem the variables may be of such a nature 
that the bipolar form of expression of the factors may appear most simple 
and clear to the analyst. An example of this is given by the solution of eight 
physical variables which is presented in Table 7.4. An eminent biologist 
selected the principal-factor form not only because it was the most statisti- 
cally elegant solution but also because the bipolar nature of the factor 
“body type" was explicitly revealed. For other problems the chief aim may 
be to secure the greatest simplicity in the factorial solution. Usually this 
may be accomplished by the choice of an oblique factor pattern, as illus- 
trated in Chapter XI. Certain data, however, such as the political variables 
analyzed in 11.5, may yield an oblique solution which is no simpler than an 
orthogonal one. For such data, then, an orthogonal solution would be pre- 
ferred. In other cases, also, if the investigator prefers a solution with un- 
correlated factors, he would select the bi-factor form including a general 
factor, or the multiple-factor form involving group factors only. 
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It should be evident that in the choice of the form of solution the analyst 
must be guided both by the statistical criteria and by other considerations, 
such as theories in given fields and the nature of the data being analyzed. 
Since the interdependence of criteria and data will vary with the type of 
problem, the choice of the analyst will vary in different situations. Such 
variations, however, are due largely to the convenience of a particular form 
of solution for a given body of data and not to any essential uniqueness of 
that solution. The faet that all solutions for a given problem may be 
brought into relationship with one another indicates that in factor analysis, 


as in other branches of applied science, several equally satisfactory laws 


may be employed as expressions of the essential nature of the phenomena 


being studied. 
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——— —— À——— ee PER UU 


APPENDIX A 
FUNDAMENTALS OF MATRIX THEORY 


It is the aim of this appendix to present in brief outline form the funda- 
mental definitions and theorems on matrices.* First, however, a short re- 
view of determinants will be given. 


1. Definition of a Determinant of Order 2 

A sum of product terms, with alternating algebraic signs, frequently oc- 
curs in mathematical work and hence has been given a special notation. 
For example, the expression 


ad — bc 
is denoted by the symbol 

a b 

e d|" 


which is called a determinant of the second order, since it contains two rows 
and two columns. 


2. Definition of a Determinant of Order 3 


The symbol 
a b a 
a2 bz Co 
a3 b; C3 


is called a determinant of the third order and stands for 
abaca — abaca + abaci — azbıCa + asbic2 — asbotr . 


A method for obtaining the expanded form of a determinant will be de- 
scribed in 5 below. ' 

The nine numbers ai, . . . , cs are called the elements of the determinant. 
In the symbol these elements lie in three (horizontal) rows and also in three 


* For a detailed treatment see Bocher, Introduction to Higher Algebra, pp. 20-80; 


Dickson, Modern Algebraic Theories, pp. 39-63. 
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(vertical) columns. For example, as, bz, c; are the elements of the third row, 
while the three b's are the elements of the second column. The diagonal 
from the upper left-hand corner to the lower right-hand corner is called the 
principal diagonal. In the third-order determinant the principal diagonal 
contains the elements a, bs, cs. 


3. Definition of a Determinant of Order n 


A determinant of general order n may be denoted by 


au Gig ... Gin 

Q2 Q22 ... Gan 
A= i 

Qni Anz ... Onn 


where the n? elements are denoted by a’s with two subscripts, the first repre- 
senting the number of the row and the second the number of the column in 
which the element appears. By definition the determinant A shall stand for 
the sum of the n! terms each of which is (apart from sign) the product of 
n elements, one and only one from each column, and one and only one from 
each row. The algebraic signs are determined most easily by the method 
of expanding the determinant which is explained in 5 below. 


4. Minors and Cofactors 


The determinant of order n — 1 obtained by striking out the row and 
column crossing at a given element of a determinant of order n is called the 
minor of that element. Thus, corresponding to the element aik, Which stands 
in the jth row and the kth column of the determinant A, there exists the 
minor M ;; which is obtained upon crossing out the given row and column. 
Frequently there is occasion to consider not this minor M ik but the cofactor 
A jx of a;, defined by 


Air = (1) M5. 


The algebraic signs attached to the minors to obtain the corresponding 


cofactors are alternately + and —, as indicated by the following diagram 
which is associated with the elements of a determinant: 


+-+- 
-+-+ 
+- +- 
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b. Expansion of a Determinant 
Any determinant A may be expanded according to the elements of any 
row or any column, that is, 


n 


A= Said je G-12,...,n, 
k=1 

A= apd (k =1,2,...,n). 
j=1 


Thus, for the third-order determinant 


au Gi2 13 
A = |an às azj|, 
Gs Q32 Ass 


the expansion according to the elements of the second column becomes 


A = andi + a22À 22 + 0324 52 
— aMi + a22M 22 — 32M s2 
—ais(anidss — 31423) + az:(ana33 — 3113) — @32(@11023 — Qm) , 


which, upon rearranging of terms, may be written as follows: 


A = audosdas + Q12023031 + dis032021 — Qi302»031 — (23032041 — Q33021012 . 


By successive applieation of the method of expansion of a determinant 
according to the elements of some row or column, & determinant of any order 
eventually can be reduced to the explicit expansion of determinants of the 


second order. 


6. Definition of a Matrix 

A system of mn numbers a;; arranged in a rectangular array of m rows 
and n columns is called an m X n matriz. If m = n, the array is called 
a square matrix of order n. A matrix will generally be represented as follows: 


Qi Gig «++ Gn 

Qo  Q22 den 
A= " 

Ami Am2 +++ Amn 


although parentheses are sometimes used instead of the double bars, or the 


matrix mi be designated by its general element in the form |la;,|| or 
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(a;x). The notation here used, matrices being denoted by bold-face type, is 
systematically followed in the text. 

Simple examples of one-rowed matrices are the notations (z, y) and 
(z, y, z) for points in a plane and in space, the elements of these matrices 
being the coordinates of the points. Similarly, the set of coordinates repre- 


senting three points in a plane, when arranged in a definite order, form a 
matrix, e.g., 


7. Distinction between Matrices and Determinants 


It should be noted that even when a matrix is square it is not a determi- 
nant. A determinant, whose elements are real numbers, represents a real 
number, while a matrix does not have a value in the ord 
difference between a square matrix and a determinant is clearly seen upon 
interchanging the rows and columns; the determinant has the same value, 
but the matrix is generally different from the original one. 


inary sense. The 


8. Transpose of a Matrix 


A matrix which is derived from another by interchanging the rows and 
columns is called the transpose of the original matrix. Thus, if 


an a» djs 
A= |an 022 Q23 
431 Q32 a33 


then the transpose of A is the matrix 


Qn G5 az 
em 
A’ = |l ais Q22 Q32 


Oi3 Q23 Q33 
The prime notation for the transpose is followed throughout the text. 


9. Determinants of a Matrix 


Although square matrices and determina; 
it is possible to form from the elements of 
which is called the determinan 
ployed is bold-face type for t 


nts are wholly different things, 
a square matrix a determinant 
t of the matriz. The notation which is em- 


he matrix and the corresponding italic type 
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for the determinant of the matrix. Thus the determinant of a square ma- 
trix A is denoted by A. Other determinants, of lower order, can be formed 
from any rectangular matrix by striking out certain rows and columns. For 
many problems it is important to know the order of the highest nonvanish- 
ing determinant of a matrix. 


10. Rank of a Matrix 
A matrix A is said to be of rank r if it contains at least one r-rowed deter- 
minant which is not zero, whereas all determinants of A of order higher than 


T are zero. 
By the rank of a determinant is meant the rank of its matrix. 


11. Singular Matrix 
A square matrix is said to be singular if its determinant is zero. Other- 
wise, it is called nonsingular. 


12. Matrix Equations 


Any two matrices A and B are said to be equal if and only if every ele- 
ment of A is equal to the corresponding element of B. Thus, if A = (aj) 
and B = (b;x) then the equation 


A=B 


implies that a; = bj. for every j and k. Thus it is evident that a single ma- 
trix equation stands for as many algebraic equations as there are elements 
in either of the matrices which are equated. 


13. Symmetric Matrix 


A matrix A is symmetric if and only if it is equal to its transpose A’. In 
other words, the matrix A = (a;r) is symmetric in case it remains unaltered 
by the interchange of its rows and columns, i.e., 


Qik = An; G, b= 1,2,...,m). 


The following is an example of a symmetric matrix: 


.78 —.16 .23 .04 

—.16 .59 —.34 —.21 

.23 —.34  .86  .40 

x .04 —.21  .40 .65 
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14. Sum or Difference of Matrices 

The sum (or difference) of two matrices each of m rows and n columns is 
defined to be an m X n matrix each of whose elements is the sum (or differ- 
ence) of the corresponding elements of the given matrices. All the laws of 
ordinary algebra hold for the addition or subtraction of matrices. 


16. Multiplication of Matrices 


The element in the jth row and the kth column of the product of a ma- 
trix A with n columns by a matrix B with n rows is the sum of the products 
of the successive elements of the jth row of A by the corresponding elements 
of the kth column of B. 

For example, if 


bu die 
Qn Qi aig 
A= j B = || bn b y 
Q5» Q22 (23 
ba bao, 


then the product C of these matrices is 


anbu + aba + Gba  aubus + Gisbo2 + arabas 


OTR um anbu + 023531 + Gba nbus +. 023022 + a23b35 


It should be noted that in this row-by-column multiplication of matrices 
the number of columns in the first matrix must be equal to the number of 
rows in the second. The product matrix then contains the number of rows 
of the first matrix and the number of columns of the second. Thus, in the 
example, the product of the 2 X 3 matrix by the 3 X 2 matrix is a 2 X 2 


matrix. This may be conveniently noted by writing the order of each ma- 
trix as superscripts, namely, 


A2*3 » Boe = C2x2 Š 
In general, 


Amn . BaXe — Qm ; 


that is, the product of an m X n matrix by an n X s matrix is an m X $ 
matrix. 


Multiplication of matrices is not commutative in general, that is, 
AB x BA. 


Hence it is important to specify in wh 
the product AB the matrix B is said to 
A is postmultiplied by B. 


at order matrices are multiplied. In 
be premultiplied by the matrix A, or 


La 
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16. Scalars 


In order to distinguish the ordinary quantities of algebra (i.e., real and 
complex numbers) from matrices, the former are called scalars and will here 
be designated in italics. The product of a matrix A by a scalar k (KA or Ak) 
is defined to be the matrix each of whose elements is k times the correspond- 
ing element of A. All the laws of ordinary algebra hold for the multiplica- 
tion of matrices by scalars. 


17. Diagonal and Scalar Matrices 


A matrix in which the diagonal elements do not all vanish and all remain- 
ing elements are zero is called a diagonal matrix. A special instance of such 
a matrix is one in which all the elements of the diagonal are identical; it is 
then called a scalar matriz. If a scalar matrix 


k O9 . 0 

O X; 2.0 
k= 

0 0 oos sk 


is premultiplied or postmultiplied by any matrix A of the same order as k, 
the following relationships become evident: 


kA = Ak = kA. 
In particular, the matrix 
io. 0 
0X. 0 
I- 
0.0 1 


is called the identity matriz, and it has the property that, if A is any matrix 


Whatever, É 
IA — AI— A. 


It is evident that, in matrix algebra, all scalar matrices may be replaced 
by the corresponding scalars and, conversely, that all scalars may be con- 
Sidered as standing for the corresponding scalar matrices. The identity ma- 
trix I corresponds to unity in ordinary algebra, and hence in products of 


matrices the factor I may be suppressed. 
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18. Inverse Matrix 


If a matrix 
Gu ie din 
O» Az ... Gan 
A= 
Qni Anz? ... Qnn 


is nonsingular, i.e., A = 0, then there exists another matrix 


An An An 
7 = Fs Ses, Bet ma di 
Sy T T S cg rs is ab Ai Ag ... Ano 
A " i 
du dm An Za Aaw asa Aa 
A udo ces J 


in which A+; denote the cofactors of the elements of A. The matrix A^! is 


called the inverse of A and is itself a nonsingular matrix which has the prop- 
erty 


AA“ = ACA =I. 


It should be noted that the rows and columns of cofactors in the inverse of 
a matrix are interchanged. In other words, when 1/A is factored out, the 


element A,, in the jth row and kth column of the inverse of A is the cofactor 
of the element az; in the kth row and jth column of A. 
19. Theorems on Transpose and Inverse of Products of Matrices 


The transpose of a product of matrices is equal to the product of their 
transposes taken in reverse order. Thus, 


(ABC)' = C'B'A'. 


The inverse of a product of matrices is the product of their inverses taken 
in reverse order. For example, 


(ABC)?! = CB-14-1, 


APPENDIX B 
MISCELLANEOUS TOPICS AND PROOFS 
B.1. Brief Description of a Set of Twenty-four Psychological Tests 


To illustrate the factorial methods a numerical example is introduced 
in 2.8. The variables from which the fundamental data (the correlations of 
Table 2.2) are obtained consists of twenty-four psychological tests which 
were given to N — 145 children of the Grant-White School of Forest. Park, 
Illinois.* Copies of the tests are reproduced in the monograph by Holzinger 
and Swineford, so that only brief descriptions of them are given here. 


l. Visual Perception Test. A nonlanguage multiple-choice test composed of items 
selected from Spearman's Visual Perception Test, Part III. Testing time: 19 min- 
utes. 

2. Cubes. A simplification of Brigham's test of spatial relations. Testing time: 
8 minutes. 

3. Paper Form Board. A revised multiple-choice test of spatial imagery, with 
dissected squares, triangles, hexagons, and trapezoids. Testing time: 8 minutes. 

4. Flags. Adapted from a test by Thurstone. Requires visual imagery in two or 
three dimensions. Testing time: 53 minutes. 

5. General Information. À multiple-choice test of a wide variety of simple scien- 
tific and social facts. Testing time: 18 minutes. 

6. Paragraph Comprehension. Part III of Traxler Silent Reading Test, Form 1, 
for Grades VII-X. Comprehension measured by completion and multiple-choice 
questions. Testing time: 20 minutes. 

7. Sentence Completion. A multiple-choice test in which “correct” answers reflect 
Eood judgment on the part of the subject. Testing time: 6 minutes. 

8. Word Classification. Arranged by M. A. Wenger. Sets of five words one of 
Which is to be indicated as not belonging with the other four. Testing time: 10 
minutes, 

9. Word Meaning. Part II of Traxler Silent. Reading Test. A multiple-choice 
vocabulary test. Testing time: 14 minutes. 

10. Add. Speed of adding pairs of one-digit numbers. Testing time: 2 minutes. 

11. Code. A simple code of three characters is presented and exercise therein 
given to measure perccptual speed. Testing time: 2 minutes. 

12. Counting Groups of Dots. Four to seven dots, arranged in random patterns, 
to be counted by the subject. A test of perceptual speed. Testing time: 4 minutes. 

13. Straight and Curved Capitals. A series of capital letters. The subject is re- 
quired to distinguish between those composed of straight lines only and those con- 
taining curved lines. A test of perceptual speed. Testing time: 3 minutes. 

* K. J. Holzinger and F. Swineford, A Study in Factor Analysis, 1939. 
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14. Word Recognition. Twenty-five four-letter words are studied for three min- 
utes. These words are then to be checked from memory on a hundred-word list. 
Testing time: 5 minutes. (Score includes two forms.) 

15. Number Recognition. Similar to Test 14. Fifteen three-digit numbers. 

16. Figure Recognition. Similar to Test 14. Fifteen geometric designs. 

17. Object-Number. Twenty pairs of names of familiar objects and two-digit num- 
bers are studied for three minutes. The words only are then presented to the sub- 


ject, who is required to supply the proper numbers. Testing time: 5 minutes. (Score 
includes two forms.) 


18. Number-Figure. Similar to Test 17. Ten pairs of numbers and geometric 
figures. 

19. Figure-Word. Similar to Test 17. Ten pairs of geometric figures and words 
studied for one minute. 

20. Deduction. Logical deduction test using the symbols ) and ( and the letters 
A, B, C, and D. Testing time: 24 minutes. 

21. Numerical Puzzles. A numerical deduction test, the object being to supply 
four numbers which will produce four given answers employing the operations of 
addition, multiplication, or division. Testing time: 14 minutes. 

22. Problem Reasoning. A reasoning test in completion form. Each problem lists 
the steps in obtaining a required amount of water using two or three vessels of given 
capacity. Testing time: 14 minutes. 

23. Series Completion. From a series of five numbers the subject is supposed to 
deduce the rule of procedure from one number to the next, and thus supply the sixth 
number in the series. Testing time: 14 minutes. 

24. Woody-McCall Mixed Fundamentals: F orm I. A series of 35 arithmetic prob- 
lems, graduated for diffieulty, is included. Testing time: 20 minutes. 


It should be noted that Tests 3 and 4 are Tests 25 and 26 of A Study in 
Factor Analysis, while Tests 3 and 4 of that monograph have not been used 
in the present study, This change has been made as a consequence of the 
findings in the other study. It was found that the original Tests 3 and 4 
were too difficult for the group of children in the sample and hence yielded 


smaller reliability coefficients than the revised forms 25 and 26 (which are 
called here Tests 3 and 4). 


B.2. Factor Patterns as Classical Regression Equations 


It will be shown here that any equation of a factor pattern may be con- 


equation, where the dependent variable is 
ndependent variables are the hypothetical 


factors. The formal expression for such an equation is 


(B.1) Zi = Bali + Bho +... H BEA + BU; 


(99 1,52: 2550) s 


| 


APPENDIXES 311 


where the subscripts on £ are so arranged that the position of any regression 
coefficient is uniquely determined. Thus, for the coefficients 8;,, the first 
subscript j indicates that z; is the dependent variable, and the second sub- 
Script s (s = 1, 2, .. . , m) shows the factor F, to which the coefficient is 
attached. The regression coefficient attached to the unique factor U; is dis- 
tinguished merely by one subscript. Of course, (B.1) cannot be obtained 
by the ordinary methods of solution for a regression equation because the 
faetors are not observed variables. It will be shown, however, that a pat- 
tern produced by a factor analysis consists of equations of the form (B.1). 

The formal expressions for the coefficients in (B.1) may be written, al- 
though the factors are not observed variables. The correlations among the 
variables involved in (B.1) may be given a symbolic representation even 
though they may not be obtainable by experiment. Let the matrix of inter- 
correlations of factors, bordered by the correlations of any variable z; with 
the factors, be denoted by A. Then 


l rue Tar, Tue, 03 uk. C ÜuUj 
Tra PRG TEES ce TEES 
TF; TPF, 1 TEE, «s+ TRF, OU 
A = || Trj TRF, TPP; 1 sce trae, 0 |, 
Trao TFQF, TFQF, TUA, 0 
TUjs; 0 0 0 jam D 1 


since the unique factors are assumed to be uncorrelated with the common 
factors. 

Let Ay be the minor of the element in the first row and column, A; the 
cofactor of the element r-u, and Aj the cofactor of the element Tug, 
(s = 1,2,..., m). Then the regression coefficients in the linear equation 


for z; are given by* 


Aj 
(B.2) Bs = Au 
and 

A; 
(B.3) Bj2-— AS 


* Karl J. Holzinger, Statistical Methods for Students in Education, p. 313. 
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The last expression can be simplified by inserting the value A; = —r;ju;An. 
It then becomes 
Bi = 250; 
or, on applying formula (2.18), 
(B.4) B; = aj. 


The algebraic solution, by determinants, for the coefficients aj, of the 
factor pattern, 


(2.45i«) 2; = aly + aja +... + ag Fu + a;U;, 
by means of equations (2.17) is* 


(2.19%) a — (-12..,ms-12,..,m. 


Now A is precisely the determinant An, and A; is readily seen to be —Ajs. 
Hence 


(B.5) Bis — 0&5. 
Formulas (B.5) and (B.4) show that the equations (2.16) of a factor pat- 
tern are exactly of the form of the classical regression equation (B.1). 

It will next be shown that the equations of a factor pattern, considered 
as regression equations, include the errors of estimate. In other words, the 
multiple correlation between the dependent variable z; and the independent 
variables Fi, F2,..., Fm, U; is unity. The standard error of estimate of 
the variable z; from the regression equation (B.1) is given byt 


B.6 Baal, 
(B.6) Ue 


Expanding the determinant A accordin 


£ to the elements of the last column, 
equation (B.6) may be written 


S - A E =r} u; An + (An + Tar An + rir Ait... + Train) 
! An An " 


which simplifies to 


(B.7) $$ 21— (rar, Bn + Tar Bi +H... +t TsrQD;) — a? 


upon substituting the values from (B.2) and Tsu; = a; 


* See p. 16. 1 Holzinger, op. cit. 
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In (B.5) the relationship 85, = aj, was deduced, and for uncorrelated 
factors Tr, = aj as shown in (2.20). Putting these values into (B.7), it 
reduces to 


(B8) S}=1- (a, +a +...+43,) a? =1—-h-a=0, 


where A is the communality of variable j, and the last equality follows from 
the fact that the communality and uniqueness sum to the total unit variance 
of a variable. The last equation shows that, in predicting a variable z; from 
the independent variables Fi, Fa, . . . , Fm, U;, there is no error of estimate. 
A factor analysis usually produces the values of a;, so that the communality 
hiis determined. The coefficient of the unique factor U; is then obtained 
merely from the relation, 


a;= V1 — Hj, 


to bring the total variance of the variable up to unity. Thus any equation 
of a factor pattern may be thought of as a classical regression equation with 
the error of estimate given by the coefficient of the unique factor. 


B.3. Indeterminateness of Factor Solutions 

It is a well-known proposition that the factor problem is indeterminate 
in the sense that, given a matrix of correlations of a set of variables, the 
coefficients of a factor pattern are not uniquely determined. That is, sys- 
tems of orthogonal, or uncorrelated, factors F; may be chosen, consistent 
with the observed correlations, in an infinity of ways. Perhaps the clearest 
demonstration of this theorem was given by Hotelling,* and that is the 
form that will be employed here. 

For simplicity of proof it will be assumed that the factor pattern con- 
tains as many factors as variables. Without specializing the type of factors 
(e.g., into common and unique), the pattern may be written 


(B.9) 2; = any + alot... + anfa (G =1,2,...,%), 


where the factors are normally distributed having zero correlations and in 
standard form, that is, zero means and unit variances.] The condition that 
the means shall be zero is expressed by 


i ZF, = 
* N 
* Harold Hotelling, “Analysis of a Complex of Statistical Variables," Journal of Edu- 
cational Psychology, 1933, Sec. 1. 
+ It will be assumed here that these properties, which are postulated for an infinite 
population, also hold for & sample of N observations. 


(B.10) Mr 0, 
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where the summation is taken over N values. The properties of zero corre- 
lations and unit variances are expressed in the single statement 
ZEF,F, 


(B.11) No T 


where ô, is the Kronecker delta, which is equal to unity if s = t and equal 
to zero if s ¥ t. 

Now the correlation between variables z; and z+, equal to unity if j = k, 
is given by 


=. - 
(B.12) rie = 2 


where the summation is over the N values of the variables z; and z;. Sub- 
stituting the value of 2; from (B.9) and a similar expression for z, into 
(B.12), and making use of the conditions (B.11), equation (B.12) becomes 


(B.13) Ti = S aran (j,k =1,2,...,7). 
s=1 


The indices j and k take on n? values together,* but since Tik = Trj the 
number of equations (B.13) is only 


nt = (3) = nt — ina- = n(n +1), 


This number of equations is insufficient for uniquely determining the n? 
quantities a;, when the correlations are given. The difference between the 
number of unknowns and the number of conditions is 


n? — n(n + 1) = 3a(n — 1). 


This suggests that it should be possible to express all the coefficients in 
terms of 4n(n — 1) of them, or generally, in terms of 
parameters. Thus systems of uncorrelated factor 
sistently with the observed correlations, 
may also be interpreted geometrically. I 
are 43n(n — 1) degrees of freedom of a rigid rotation. Thus after a set of 
n factors is selected there are 3n(n — 1) degrees of freedom of a rigid rota- 
tion of this set into another set of factors. 


One assumption made on the nature of the factors F has not been brought 


3n(n — 1) other 
l s F may be chosen, con- 
in $n(n — 1) ways. This result 
n a space of » dimensions there 


* It is here assumed that the diagonal values of the correlation matrix are unity. 
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explicitly into the proof. That assumption is that the factors are normally 
distributed. It is because of this condition that no additional equations for 
determining the a;,’s could be obtained from the moments of higher order, 
or of other parameters of the population. For the z;'s, being linear combi- 
nations of normally distributed factors, have a multivariate normal dis- 
tribution; and every parameter of such a distribution is a function of the 
means, standard deviations, and correlations.* Thus all the available in- 
formation is embodied in the equations (B.13), and these being fewer in 
number than the coefficients ajs, the indeterminacy exists. 


B.4. Basic Axioms for Euclidean Geometry 
The most fundamental axioms are those of incidence or connection. They 
may be listed as follows for three-dimensional space: 
1. Any two distinct points uniquely determine a straight line. 
2. If P, and P» are distinct points, there is at least one point not on the straight 


line PVP». 
3. Any three noncollinear points determine a plane. 
4. If two distinct points P1, P» both belong to a plane 7, every point of the straight 


line PP belongs to 7. 
5. If P, Pa, Ps are noncollinear points, there is at least one point not on the plane 


P,P2P3. 
6. If two planes have a point P, in common, they have a second point P: in com- 


mon, and hence the line P,P» in common. 


Axioms 2 and 5 postulate the existence of a two- and three-dimensional 
space, respectively. If Axiom 6 is assumed, it limits space to three dimen- 
sions. On the other hand, if at least one point not in this region is postulat- 
ed, then a four-dimensional region is obtained. 

There is some difficulty in determining all the elements of space by means 
of the preceding axioms; for, while Axiom 1 postulates that two distinct 
points determine a line, there is no axiom guaranteeing that two distinct 
lines in a plane determine a point. This, of course, is not true in the Eu- 
clidean plane since parallel lines have no point in common. To simplify the 
geometrie discussion, therefore, the following projective aziom will be as- 
sumed: 

7. Any two distinct straight lines in a plane uniquely determine a point. 


This axiom extends a Euclidean to a projective plane. The advantage of 
working in a projective space is that most theorems are greatly simplified 
by not having to treat the special cases of parallel elements. 


* Yule and Kendall, An Introduction to the Theory of Statistics, 1937, Chap. XIV, 
Sec. 28. 


316 FACTOR ANALYSIS 


B.b. Proof of Theorem 3.1 


For purposes of reference, the theorem is here repeated: J f m is the rank 
of the matrix 


Yu fg Tg ... By 
Wap Ta Ta ... Tay 
lezl] = Tar Ta Ta ... m 1 
Tn Tn Tu... Lan 
the points Py, Ps, . . . ; Pn are all dependent upon m of them, which are them- 


selves independent. 

The proof of this theorem may be split into two parts. First consider the 
case where n S N. By hypothesis the matrix |; is of rank m, so that 
without loss of generality it may be assumed that the determinant 


Tu Tiz Tim 
| 
Too Xo T: 
D= 2m 
Tmi Ime Taim | 


is different from zero. If m = n the set of equations 


n 
Diui (gm du E 
k-i 
have the unique solution lh-—i1-.- én = 0, since D = 0. Then ac- 
, s , 


cording to the definition (3.2), the points Py Papan i s P. are linearly inde- 
pendent. If m <n the pomis Py, Phs gy Pa may be shown to be inde- 


pendent by the preceding argument. This establishes the last part of the 
theorem. 


Let Di, Da, ... , Dmi be the cofactors of ty, 9225... j Tmi in the matrix 


Tuo iz .., Tim Tii 

Ta Ta ... Tom Tz; 
Eos Š , 

Tmi Lm Imm Lm; 


Tpm Tpi 
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where p = m 4- 1,..., n and i is arbitrary. The determinant of this ma- 
trix, when expanded according to the elements of the last column, becomes 


(B.14) muDi + gaDs +... + tmiDmi + zuD. 


This expression vanishes, for, if 7 € m, two columns then have equal ele- 
ments; and, if 7 > m, it vanishes, since the rank of |lz;;| is m and every 
(m + 1)-order minor vanishes. The solution for zp: from the expression 
(B.14) set equal to zero is 


(B.15) tpi = > lates (p-m-1...,m, 
q=1 
where the constants 
Dai 
= D 
do not depend on the elements zi; Toi, . . . , Tmi Tpi. It follows from defini- 
tion (3.1) that the points Pp, whose coordinates are given in (B.15), are 
linearly dependent on the points Pi, P», . . . , Pm, which are themselves in- 
dependent. 
If n > N, consider the points P;: (ri, 2j... , tiv, 0,...,0) in the 


space of n dimensions. Then the foregoing argument can be applied to 
obtain the relation (B.15), and thus the theorem is established for all 


values of n. 


B.6. Proof of Theorem 3.4 

The product of the matrix [[z;;|| of standardized values by its transpose 
is equal to the correlation matrix ||r;;|| multiplied by N , as shown in equa- 
tion (3.44). Since the theorem is concerned only with the ranks of the 
matrices, the non-zero factor N is irrelevant to it. The product of the two 
matrices might be represented by |||] = ||Nr;«|| to be absolutely rigorous, 
but for simplicity ||rf,| will be written ||r;.||. With this clearly in mind, 
the theorem may be restated as follows: If m is the rank of the matrix 
llz;:[|, and if [[z;l| is its transpose, then the rank 4 of [ril = llzsll + Ilzi] 
is equal to m. 

The proof consists of two parts: first, to show that u S m and, second, 
that m < u. That the rank of the product of any two matrices cannot ex- 
ceed the rank of either factor is a well-known theorem. The remainder of 


1 See, e.g., L. E. Dickson, Modern Algebraic Theories, p. 51. 


318 FACTOR ANALYSIS 


the proof is concerned with showing that m < u for the product of a matrix 
by its transpose. 


Let 
Zu Ziz Zim | 
Zo» 222 Zom 
zd = 
Zml £m? Zmm 


be a minor of |[z;;|| whose rows form a maximal linearly independent set of 
rows of |lz;;||. By this is meant that the rows of |l2pq|| are linearly inde- 
pendent and every minor of which |jz,,|| is a proper minor has its rows 
linearly dependent. Then the rank of ||z;l| is the number, m, of vectors 
(or rows) in this set. 


The product of ||zp,|| by its transpose, |lz,, l|, may be represented by 


«lzel, 


where ||r,.|| is an m-order minor of ||r;l]. Now, if* 


IIrpel| = llo] 


E 
Tu Tie $*e Tig Zo 
Tol T22 ... To 
(B.16) "d. =i 
Tmi Tmo Trim 
Zm 


then, premultiplying by ||zı zs . . . zmll, this expression becomes 


Zi 
Zo 
0 = la2:.. . 2mll + Irall - 
(B.17) 2 
Zm 3 
2 
= n2... zal] « Zpall + Ilall - 
Zm 
* This condition is equivalent to (3.2), and if it implies that zı 2-2, =... =z, = 0, 
then the columns of the correlation matri ^ à 


x are linearly independent. 


d 
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Let the product of the last two matrices be represented by the column vec- 
tor [y;], as follows: 


yi E 
y» Zo 

= dl 1]. 
Ym 2m 


then the transpose of [y] is 


lal = ligo - - Yall = lazo semel + Meal - 
Equation (B.17) may now be put in the form 
yı 
Y2 
m 
0 = lil © {ye} = lys vnl] | = Doe 
; = 
Ym 


In order for a sum of positive numbers to be zero, each element must be 
zero, and therefore ||y,l| is a zero vector. Then 


llvall = Maze ~ - - 2mll ool = 0, 


and, since the rows of ||zpql| are linearly independent, it follows from the 
definition (3.2) that z1 = z» = ... = Zm = 0, that is, the vector ||z1 22 . . . 2ml| 
is a zero vector. It has thus been shown that the condition (B.16) implies 
that 2: = zp =... = 2m = 0, and hence according to (3.2) the columns of 
\|rps|| are linearly independent. Then the matrix \|rp.|| is nonsingular, and 
its rank is at least equal to its order, m. Now that » = m and, as has been 


pointed out, » X m, it therefore follows that u = m. The rank of the prod- 
uct of a matrix by its transpose has thus been shown to be equal to the com- 


mon rank of either factor. 

B.T. Standard Error of a Function in Terms of the Standard Errors of Iis 
Variables 

Let f(zi, £2, . - 

let f; = af/óz;, e; = 9zp Tik = zjaw 


df = fidzi + fadzs + ... + fndin .- 


., Zn) be a function of the n variables zi, T2, ... , Tn and 
Then the total differential is given by 
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Squaring this expression and taking the mean value over the population 
produces 


of — fici t f?o$ +... + Sioh + 2(fifsrioorio 
+ fifaro +... + fifaoiontin + fofsoocates 
Paes + fofno2tnfon +... 3I i E nO qe AORTA )y 


n n 
x = Jig + 2 M fiie 
j=1 


j<k=1 


(B.18) = > Sifkojokrik, 


since rj; = land rj, = rij. " 
For the particular case in which the variables 7j To, .. . , 2, are statisti- 
cally independent (i.e., uncorrelated), formula (B.18) reduces to 


(B.19) ej = >> fh. 
j=1 


À very important special case of (B.18) occurs when f is a function of a 
single variable z. Then 


2 
(B.20) oko = (2) á. 


This states that the mean Square error of a function of z is equal to the mean 
Square error of z multiplied by the Square of the derivative of the function 


with respect, to x. It is this theorem which is employed in obtaining (6.25) 
or (6.26). 


B.8. Mean Value of Certain Ratios 


The mean value of a nonlinear functi 
general, only be expressed in terms of t 
correlations of the original variables to 
required to find the mean value of the 
for a, b, and c. The determination of 
follows a similar pattern to the one whi i 


save space, therefore, their derivations will not be given. 
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Denoting the mean over v = P $ ?) values by scoring, it is required to 


find an expression for Z(rra/rj;; J < k, j,k = e, jk = 1, 2,...,m) Jv, 
or employing the definitions (6.19), 


(B.21) rj "x 


e y a c 


where the summation extends over the v values of the ratio. Let a, 8, and y 
stand for the respective deviates of a, b, and c from their means, that is, 


a=a-4d, B-b-b, y-2c-—6. 


Then (B.21) may be expressed as follows: 


GEGE 


Now expand the last bracket by the binomial theorem, dropping third and 
higher degree terms. To this approximation, the formula becomes 


ab\ _ abl £424 %)(1-37 463) 
(S)-82 D0 +545+8)0 el ie 


This may be expanded, again keeping only terms to the second degree, as 
follows: 


(8) - 21 (e ize 528 + 5 oen 
v ab 


cà 


Employing the definitions of the standard deviation and the correlation 
coefficient for these variables, the mean of the required ratio finally re- 


duces to 
— 6o? 
(B.22) (S) -5 (1 4 sera, Beets _ Bede 2 ) l 
C 


^6 ab ác bc G 


The last of equations (6.28) is precisely (B.22). The means of the other 
ratios in (6.28) can be obtained in a manner similar to that which is used 
here for ab/c*. It must be remembered that in the derivation of the formu- 
las (6.28) it was assumed that a/à, 8/b, y/c are so small, in each case, that 
powers higher than the second can be neglected. 
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B.9. Derivation of the Principal-Factor Method 


The first stage of the principal-factor method involves the selection of the 
first-factor coefficients a; so as to make the sum of the contributions of that 
factor to the total communality a maximum. This sum is given by 


"(Qr 306) Ai — aj d al +...+03,, 


and the coefficients aj, must be chosen so as to make A; a maximum under 
the conditions 


(7.4005) Ti = M ajar 6, 5-12, ..,n), 
t=1 


where 7;; is the communality h? of variable z;. 

In order to maximize a function of n variables when the variables are 
connected by an arbitrary number of auxiliary equations, the method of 
Lagrange multipliers* is particularly well adapted. This method will be em- 
ployed to maximize A;, which is a function of the n variables aj, under the 
3n(n + 1) conditions (7.4) among all the coefficients aj. Let 


n NET 
(B.23) 2T = A; — D uarn =A= D M anajas, 


j,k=1 j,k=1 t=1 


where yj; (=p,;) are the Lagrange multipliers. Then set the partial deriva- 


tive of this new function T with respect to any one of the n variables aj 
equal to zero, namely, 


oT Z 
(B.24) Jan =a; — Z rnan =0, 


and similarly put the partial derivative with respect to any of the other 
coefficients a;, (t = 1) equal to zero, that is, 


oT S 
(B.25) m È unan =0 (= 1). 


The two sets of equations (B.24) and (B.25) may be combined as follows: 


oT Z 
(B.26) Jaj, ^ hen — M usan =0 ([51,2,..,,m), 
k-1 


where ôu = 1 if t = 1 and à, = O if t = 1. 


* See, e.g., William F. Osgood, Advanced Calculus, 1932, pp. 180-85. 
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Multiply (B.26) by aj; and sum with respect to j, obtaining* 


n n n 
(B.27) M aj — I M enanar ep; -$ 
=1 j=1 k=1 


j= 

à A 
Now, the expression X unan is equal to am according to (B.24), and, 

j=1 


setting Sa = X, equation (B.27) may be written as follows: KA 
je UN 
(B.28) os — S anan = 0. 


kel 


Upon multiplying (B.28) by a;,and summing for f, this equation becomes 


(B.29) aiñ — Maa (> aran) =0, 


k=1 t=1 


or, by using (7.4), 
n 
(B.30) Y rnan — Mag = 0. 
k=l 
The expression (B.30) represents n equations, one for each value of j. 
Writing these equations explicitly, there results 


(hi — Man + Ty21 HTa +... Tru = 0, 

Taan + (h3 — M) — bre t... Tora = 0, 

(B.31) Tatu + T3202 + (A5 — Ay)agy +.. + Tanan = 0, 
Tati + TnoQo1 Torn d.d RR adm = 0. 


These equations furnish the basis for the solution of the unknown coeffi- 
cients aj. The remaining description of the method is presented in 7.3. 


B.10. Direction Cosines of Oblique Reference Axes 
One stage in the process of obtaining an oblique factor solution involves 
the calculation of the direction cosines of these reference vectors with re- 


* 'This procedure was suggested by M. A. Girshick, “Principal Components," Journal 
of the American Statistical Association, 1936, pp. 519-28. 


oe 
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spect to the original orthogonal reference system. In 11.3 two methods were 
suggested for the determination of such direction cosines. One method is 
based upon averages of points representing the variables, and the other de- 
pends upon composite variables. It will now be shown that these two ap- 
proaches lead to identical values of the direction cosines. 

For simplicity, let there be two initial orthogonal factors, F, and F;, and 
eight variables consisting of two subgroups. Let the composite variables of 
these subsets be defined by 


Vu —zdz + 23 + n, 
Ve = Zs + Z% + 27 + zs. 


Then the coordinates of the point representing the first of these variables 
are given by 


4 
d 1 
(B.32) Pi: (s. = = Dre, i Twp, d. Srv) i 


j=l 


and a similar expression is possible for the second variable. The distance 
of this point from the origin is 


(B.33) D(0P)) = Vr, + rhe, 
and the direction cosines of the line from the origin through this point are 
Ty F Tv F, 
B.34 Air = EL 
( ) u DOP) and Aor DOP, ` 


Instead of employing the composite variable, the centroid of the first 
subgroup may be obtained, as follows: 


4 


1 ix 
(B.35) Qi: (re , i2) j 
gat jal 
Now it may be noted from (B.32) that 
4 4 


` Tir, = OTF,» 1 Tie, = Ou Tor. 


P io 
j=1 j=1 


and hence the distance of the point Q, from the origin is 


(B.36) D(0Q) = tos, Vrt r, + 1, = ło, D(OP,) . 


v 
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Then the direction cosines of the line OQ; are given by 


H 
aTr oF, To,F, 


Jo DOP) DOP) ~ "V 


1 
lewrsr, Tug 


mem ie,D(0P) — D(0P) 


= M and 


From (B.34) and (B.37) it is evident that either the method of composite 
variables or the method of averages yields the same values of the direction 
cosines of the axis passing through the first cluster of points. Similar results 
can, of course, be obtained for the axis through the second cluster of points, 
and the method can readily be generalized to any number of variables in the 
subgroups and any number of factors. 


B.11. Matriz Formulation of the Relation between a Pattern and a Structure 


In matrix notation a factor pattern expressing the N values of each of the 
n variables in terms of the factors may be written in the form: 


(B.38) Z-MF, 
where 
Fy Py Fw 
Zu 7 Zin s w;x S5 eee €x 
ee Ži epo. BON F= Fmi fae was n l 
Eey sea s» Un Us ... Uw 
Zu Zn Zan e X 
Um Une Uan 


lau ie Am M 0 0 

Qs a22 am O ae 0 
M = 

du d zs dw 0 0 an 


Postmultiplying both sides of (B.38) by F’, and multiplying by the scalar 


1/N, this equation becomes 


QM Im 
(B.39) lzr = u(y FF’). 


326 FACTOR ANALYSIS 


The expression in the left-hand member of equation (B.39) can easily be 
shown to be the factor structure 


tu hie hs a 0 0 

1 DR ta tog tom 0 ae 0 
Nm s TI ur aam 

la tng CE ar ees 0 0 ... Ay 


where tj, = 7;r,, and a; = T2,u; according to (2.18). The expression in the 
right-hand member of (B.39) can also be simplified by means of the relation 


|| DF}; oe. DFF ai ZFyuU; cus EPFuU 
Lip a LBP... BPR, EZFUs ...2 EEn Uni 
NU TN|zu. ... LUiF ns ZU e SEU. 

ZUsFw o... ZU alms ZUQUs ... BUR 


Since all factors are in standard form, each element in this matrix represents 
a coefficient of correlation multiplied by N. Hence, the preceding expres- 
sion is equal to the matrix of correlations among the factors, namely, 


(B.41) Hl =s I$ rie 


where the matrix of correlations among the common factors is given by 


1 


TPP, TRF, 
6 TF F, 1 TP Rm 
TrAF, Us A 1 


Now, substituting (B.40) and (B.41) into (B.39), 


the latter reduces to 
(B.42) 


S=Mé, 


ng all the factors. This re- 
lationship shows clearly that i i 
identity matrix) then the 


E 
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sponding elements of the pattern. The explicit expression for the pattern 
matrix can be obtained from (B.42) by postmultiplying both sides by $-1, 
The result is 


(B.43) M = S&-, 


Sometimes it is convenient to have the relationship between a pattern 
and structure for the common-factor portion only. If f is used to represent 
the matrix of common factors, A the matrix of common-factor coefficients, 
and T the common-factor portion of the structure, then by analysis similar 
to the preceding it can be shown that i 


(B.44) T= Ap: * 


This relationship between the common-factor portions of the pattern and 
structure is analogous to that for the complete solution. 


APPENDIX C 
OUTLINE FOR COMPUTATION OF A BI-FACTOR PATTERN 


A bi-factor solution can be obtained from a given matrix of correlations 
by the following caleulations. This is the method developed in 6.4 and re- 
duced to a routine procedure. For concreteness the method will be de- 
scribed in relation to the computation of the pattern for the data of 2.8. 
The detailed procedure is presented as a series of steps.* 

1. Determine the grouping of variables by the method of B-coefficients, 
as developed in 2.7, for the given matrix of correlations. The method of 
computing B-coefficients is completely exhibited in 2.9 and will not be re- 
peated here. For the given data, the B-coefficients are computed in Table 
2.3, and the groups of variables indicated in (2.35). 

2. Arrange the matrix of correlations so that the variables constituting 
a group are together. Of course, only the correlations below the principal 
diagonal need be written because the matrix is symmetric. The variables 
in Table 2.2 have been arranged according to their respective groups. It 
may be convenient, but not at all necessary, to renumber the variables at 
this stage from 1 to n. In the present example the variables are already 
numbered consecutively from 1 to 24 when they are arranged by groups. 

3. As a result of the B-coefficient analysis, there are postulated six fac- 
tors, viz., the general factor and five group factors, one for each group of 
variables. This hypothesis may be changed if the ensuing analysis indicates 
necessary modifications. 

4. In the following analysis it is assumed that the matrix of correlations 
contains only essentially positive entries. This condition has always been 
found ‘to be met with psychological data; but high negative correlations 
might occur for different variables. If there are one or two negative corre- 
lations which are insignificantly different from zero, they may be set equal 
to zero for purposes of computing the general-factor coefficients. On the 
other hand, when an observed matrix includes significantly high negative 
correlations, the variables producing these values may be changed in direc- 
tion in an attempt to reduce the matrix to essentially positive correlations. 
In the example there is one negative correlation, namely, r19 = —.075, 
which is insignificant, being only nine-tenths of its standard error. Hence 
a value of zero is used for this correlation in the following step. 

5. To obtain the general-factor coefficients, formula (6.14) is used. The 
component parts of this formula will be calculated piecemeal, however, to 
clarify the procedure. First, add all the correlations of any variable s (seG:) 


* The essential steps in this outline are based upon those of Miss Frances Swineford 
given in Student Manual of Factor Analysis. 
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with each variable j of a group Gu (u #7, u = 1, 2,...,5). This sum 
may be written 
Tan 


JeGu 
where s is a fixed variable in G;, and G, is any other group. The number of 
such sums for n variables and m groups is 


X n-p) = mn — n = n(m — 1), 
izi 


where p; is the number of variables in the group G;. In the example there 
are 24(5 — 1) — 96 such sums. These sums are obtained from Table 2.2 
and are presented in Table C.1. Looking down the G, column, the first 
entry is 

*n = 7a + 152 + 753 + Tsa = 1.080. 

je 
Similarly, any other entry represents the sum of the correlations of the 
variable in the stub with all the variables in the group indicated at the head 


of the column. 


TABLE C.1 
SUMS OF CORRELATIONS FOR VARIABLES WITH GROUPS 
Gi G: Gs Gi Gi 
1 1.903 
2... 1.389 
uE 1.208 
4.. 1.651 
5 2.012 
E 1.948 
Gs} T.. 2.003 
‘| 8: 2.035 
9... 2.123 
1.531 
n- 1 
ena... 1.558 
(EMT 1.685 
1.274 
(i5. 1.171 
T 1.641 
pers 1.321 
B or 1.652 
19. 1.517 
19.. 
.293 | 2.157 .862 | 1.692 
2. j 189 1.523 | 1.584 | 1.624 
G,22....| 1.275 | 2.063 “936 | 1.704 
*22--| p540 | 2.276 | 1.245 | 1.720 
oa | "edi | 2.102, | 3-725 | 1.896 
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6. Using the values below the diagonal of Table C.1, compute the fol- 


lowing intergroup sums: 
> : > Tsj; 
SG; jeGy 


that is, the sums of all the correlations of the variables of one group with 


Y = m(m — 1)/2 such 


sums for m groups. For the example the ten sums appear in Table C.2. The 
first entry is 


those of any other group. There are evidently ( 


z(z) = 1.080 + 1.164 + 1.019 + 1.262 + 1.030 = 5.555, 
aeG2 \ jer 


while any of the others is similarly obtained. The sum in the parentheses 
was obtained for each s, in step 5, and now the summation is taken for s 
varying over a specified group. The values above the diagonal in Table C.1 
should be used to check the entries of Table C.2, but the sums need not be 
written twice in the latter table. For example, the first entry in Table C.2 
could also be obtained as follows: 


z(-) = 1.618 + 1.028 + 1.304 + 1.605 = 5.555. 
seGi \ jeGz 
Thus a check on all the additions is provided. 


TABLE C.2 


SUMS OF INTERGROUP CORRELATIONS AND DENOMINATORS 
FOR GENERAL-FACTOR COEFFICIENTS 


Gi Gi Gi a Gs 

OE MP Nes oy asus costes cansa hat cas a 
Chaser cause Bebo: tt ross ai 
( o NNNM 3.066 522. rss ccarecars 
rc MN 4.764 TOI |^ 5.200 diis 
(c NR 6.241 10.121 8.576 

Total... 19.626 28.039 | 20.387 | 26.138 31.280 | 25=125.470 

S= 62.735 

Denominator] 43.109 34.696 42.348 36.597 31.455 | 3S=188.205 


7. It will be noted from (6.14) that the denominator for the calculation 
of the general-factor coefficient for variable Zs 


correlations exclusive of the group which inc 
for all variables of a group are therefore alike. 


is the sum of all intergroup 
ludes z,. The denominators 


E C mm 


—À 
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; 8. Add the entries in the row and column corresponding to each group 
in Table C.2 and record the sum in the “Total” row. Thus, for the group 
G4, say, the Total is obtained as follows: 


4.764 + 7.091 + 5.707 + 8.576 = 26.188 . 


9. The sum of the “Total” row should then equal twice the sum, S, of 
the entries in Table C.2. In the example the sum of the entries is S = 
62.735, and the sum of the “Total” row is 125.470, or 2S, completing the 
check. 

10. Subtract from S each Total in turn, entering the difference in the 
“Denominator” row. For example, the denominator for the calculation of 


the general-factor coefficient for any variable in G; is 


62.735 — 19.626 = 43.109 . 


11. There is another computational check available at this point. Each 
Denominator is obtained by subtracting a Total from S. Hence, if there 
are m groups, the sum of the “Denominator” row is equal to mS minus the 
sum of the “Total” row. Since the sum of the “Total” row is always twice 
the sum of the entries in the table, it follows that the sum.of the “Denomi- 
nator" row must be (m — 2)S. In the example, the sum of the Denomina- 
tor" row is 188.205, which checks with (5 — 2)8. 

12. Tables C.1 and C.2 may now be employed to get the general-factor 
(Bo) coefficients. Each aĝo, or 72,5, 158 fraction whose numerator is the sum 


of the a *) = (3) = 6 products by pairs of the m — 1=5-1=4 
entries in row s of Table C.1, and whose denominator is the Denominator 
in the column of Table C.2 for that group which includes variable z,. 

13. The calculation of the numerator of a2), when there are five or more 
groups, may be shortened with the aid of an additional table. It will be 
noted that the first entry in a given row of Table C.1 is to be multiplied by 
each of the remaining entries in turn. Construct Table C.3 so that the first 
entry in any row is the sum of the m — 2 = 3 entries beyond the first entry 
of the corresponding row of Table C.1. Similarly, the second entry ina 
row of Table C.3 is the sum of the m — 3 = 2 remaining entries 1n the 
corresponding row of Table C.1. The other entries of Table C.3 are ob- 
tained in like manner, the final value for each row being the same as that 
in Table C.1. The total number of entries in each row of Table C.3 is one 
less than that in Table C.1. 

headed, and the entries recorded, 


14. The columns of Table C.3 are so n ; 
that the sum of the products of the entries for variable z, in Table C.1 by 


the corresponding entries in Table C.3 gives the numerator of aio. Table C.3 
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may then be folded for each row and placed over Table C.1 so that corre- 
sponding entries are directly under one another. The sums of the products 
of these corresponding entries give the desired numerators. 


TABLE C.3 


FOR THE CALCULATION OF THE NUMERATORS FOR 
GENERAL-FACTOR COEFFICIENTS 


Gi Ga Gi Gi 
4.721 3.494 1.903 
G 2.858 2.267 1.389 
i 3.055 2.503 1.298 
3.437 2.741 1.651 
4.583 — 3.369 2.012 
4.403. — esanceixcm 3.473 1.948 
4.208 irs ane seitens a | 3.204 2.003 
e| 4.788 to 6) i SUNL E 3.537 2.035 
| 4.472 gra ure v s.d 3.629 2.123 
3.992 2.777 
4.983 3.474 
3.593 2.718 
4.753 3.080 
3.442 2.122 
2.952 1.877 
3.675 2.479 
3.682 2.444 
4.038 2.991 
3.585 2.370 
(20. 4.711 2.554 I. 
'21. 4.731 3.208 T. 
6,122 4.703 2.640 I. 
23.. 5.241 2.965 LR focis psarna 
24. Luis 5.653 3.551 1.836 


15. When working on a caleulating machine, these numerators need not 
be copied down but can be directly divided by the denominators, which are 
given in Table C.2, to get the squares of the coefficients. For example, 


1.605(3.437) + .696(2.741) + 1.090(1.651) 
AME 
a, Hie = .21396. 


Record the squares of the coefficients in Table C4. 

16. Without the short cuts described in this outline, each a2, of the ex- 
ample if computed directly by formula (6.14) would involve 121, 134, or 149 
products in the numerator and the sum of an equal number of correlations 
in the denominator, depending on whether the variable z, is included in a 
group of 6, 5, or 4 variables, respectively. The method described in step 12 
reduces the large number of products for each numerator to 6, while the 
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method of step 13 reduces this number to 3. Of course, a certain amount of 
time is required in obtaining the sums of Tables C.1 and C.3, but the eventu- 
al saving of time is tremendous for a large number of groups. 

17. The general-factor coefficients are the square roots of the values cal- 
culated in step 15. These coefficients also appear in Table C.4. In practice 
these coefficients need not be written down in this table but can be put d 
rectly in the final factor pattern shown in Table 6.8. 


TABLE C.4 
GENERAL-FACTOR COEFFICIENTS 


a aio an To aio cry 
. 34687 .589 .33162 .576 
.12752 .357 .15036 .388 
.16074 .401 .12298 .951 
.21396 .463 .24622 .496 
. 33923 .582 .17804 .422 
.33044 .575 .26573 .515 
.28563 .584 .19544 .442 
.98978 .624 .41516 .644 
.31308 -560 .41594 .645 
.15036 .388 .41448 .644 
.27191 521 .53921 . 734 
. 16323 .404 .50705 .712 


18. Now the general-factor residuals must be obtained, and then the 
group-factor coefficients can be calculated. To facilitate the calculation of 
these residuals, a table of products of general-factor coefficients may be pre- 
pared. This table of ajoaxo is presented in Table C.5 for the given data. A 
convenient procedure for the calculation of the elements of Table C.5 is to 
proceed by columns, as follows: multiply aio = .589 by every other aio 
(k = 2,3,..., 24), in turn, to obtain the first column of values of Table 
C.5; then multiply az = .357 by all remaining aio (k = 3, 4,..., 24) to 
get the second column of values of Table C.5; and so forth for all remain- 
ing columns. 

19. Subtract the values in Tabl 
Table 2.2 to get the general-factor residuals. 
the principal diagonal in Table C.6 (the meaning 
prineipal diagonal will be clarified in step 28). Thus the gene 
ual for variables 1 and 2 is 
— idas = .318 — .210 = .108. 


e C.5 from the corresponding entries of 
Record these residuals below 
of the values above the 
ral-factor resid- 


Tig = M12 
The correlation ri; = -318 comes from Table 
.210 comes from Table C.5, and the differenc 
is entered in Table C.6. 


2.2 while the product @10@20 = 
e .108 is the residual 712 which 


TABLE C.5 
PRODUCTS OF GENERAL-FACTOR COEFFICIENTS: Ajoko 


1 2 3 4 5 6 7 8 9 2 11 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 


-269|.... 
- 266). 
- 247). 
. 289). 
- 259). 


. 180}. 
241]. 
- 187}. 
- 267]. 


- 180}. 
. 163}. 
- 230}. 
- 195]. 
. 238}. 
- 205). 


- 298). 
- 299). 
- 298}. 
- 340). 
- 330). 


PEE 


cee 


TABLE C.6 
RESIDUAL CORRELATIONS^ 


— 020|. 


LUN 
5|— .091} 


MN 
fi 


a The values in italics are not final; they are factored further, and the final residuals corresponding to them appear above the principal diagonal. 
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20. A check on the subtractions involved in the calculations of the residu- 
als is provided by means of the table of products. The general-factor re- 
sidual for any two variables z; and z+ is given by 


jk = Tj — Qjoüxo , 


and the sum of all such residuals for j fixed and all k > j is 


n 


n n 
> tik = > Tik — > AjoAko - 


k=j+1 k=j+1 k=j+1 


In other words, the sum of the entries below the principal diagonal of any 
column, j, in Table C.6 is equal to the difference between the corresponding 
sums in Tables 2.2 and C.5. Record these sums in Table C.7 and check that 
the sums obtained from Table C.6 are actually the required differences of 
sums from the other tables. 


TABLE C.7 
CHECK ON CALCULATION OF RESIDUALS 
Test 24 24 24 

j P PX P» 

ssn spain 7.528 7.056 472 
Disa ar raed 4.433 4.150 .283 
[ NS 4.589 4.501 .088 
Mass vocem 5.042 4.981 .061 
A 7.162 5.922 1.240 
Dos 6.396 5.519 -877 
at 5.512 4.840 .672 
CREDE. 5.320 5.266 .054 
CREE 4.472 4.415 .057 
JO. eor ns 4.254 2.907 1.347 
Dl peace ge 4.437 3.632 .805 
Te estan sd 3.230 2.655 .975 


21. If the grouping of variables is reasonable, the general-factor residual 
correlations of Table C.6 will all tend to be insignificant except those within 
the groups. Thus for Gi, the intercorrelations are consistently positive and 
in general greater in value than the remaining correlations with the Gi 
variables, while the residual correlations of the variables in G, with the 
others are all insignificant. The same is true for G; and G4. The group G; 
also has this property with one exception—the value 71,34 = .255 is sig- 
nificantly different from zero (shown in 6.10). The group Gs, however, con- 
tains a number of negative residual correlations and very small positive 
ones, clearly indicating that no additional factor is required for these vari- 
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ables. The general-factor residuals among the variables of G; are insignifi- 
cant and may be considered as final residuals. 

If the analysis of the table of general factor residuals indicates numerous 
revisions of the original plan, it may be expected that the general-factor 
coefficients will change somewhat. A more accurate solution may then be 
obtained by recalculating the general-factor coefficients from the revised 
plan. In the present example the modifications are so slight that such a pro- 
cedure is not deemed necessary. 

22. To obtain the group-factor coefficients, formula (6.16) is used. The 
coefficients of B, are calculated in Table C.8, where all the triads are put 


TABLE C.8 
CALCULATION OF THE COEFFICIENTS OF Bı 


Test 1 Test 2 Test 3 i TEST 4 
jk tjk jk tjk jk tik jk tjk 
23 1037 || 18... .. 1325 || 12. «s 2691 || 12 .1174 
24 3240 || 14......-- 0360 || 14......-- 1019 || 13 .1390 
D MNT D 2137 || 94.55 as os 0950 || 24. ....... 3186 || 23 .0445 
Ej ye aiats. er $2388. || ah. o m w 0812 || a$..-. 2299 | a 1003 
pues d S4 i| iaces gv DRE Il iugi exse 479 | aa os 317 


ase, and the square root ex- 


down explicitly, the average obtained in each ¢ i 
wee — 4, so that there are just 


tracted. The number of variables in Giism 


H = 3 triads for the calculation of each coefficient. Thus, the first coeffi- 
cient is obtained as follows: 
a, = b(t + tz + tsa) 
1 /.108 X .167 .108 X .195 , .167 X zm) 
3 ( da t 65 +419 


= 3(.1037 + 3240 + .2737) 


m = 2338, 


and x 
an = V.2338 = 484. 


2 and B; are c 
leulation 


i fB omputed in a similar manner in 

ine d a rr In the ca of the B; coefficients, there are 

N = ‘) = 6 triads in each case, whereas f 
2 


or each of the Bs coefficients 
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there are again only 3 triads since there are 4 variables in Gs. The signifi- 
cant general-factor residual between Tests 10 and 24 will be treated in step 26. 


TABLE C.9 
CALCULATION OF THE COEFFICIENTS OF B, 
Test 5 Test 6 Test 7 Test 8 Test 9 
jk Uk jk tik jk tjk jk tik jk tjk 
"— 3452|| 56.....| .4989| 56.....| .1239| 56.....| .5422 
V via 2243|| 58.....| .4589) 57.....| -1782| 57.....| .4442 
— 2834|| 59.....| .3354|| 59.....| .2090|| 58.....| .3379 
pps 2438|| 68.....| . 7065| 67.....| .1158| 67.....| .3646 
ET 4215] 69..... .4086|| 69.....| .0784|| 68.....] .4270 
void 3599| 89.....| .6033| 79.....| .1356| 78.....| .2470 
ah.. -..| .8130) at, .5019}| a2). 1405]| a}... 3938 
ib. asia 559 || an..... .708 || ag..... 375 || an..... 628 
TABLE C.10 
CALCULATION OF THE COEFFICIENTS OF B; 
Test 10 Test 11 Test 12 Test 13 
jk tik jk tik jk tjk 
Bl; e TEETE -5537 10, 11.. 
Yi; 185. cue .2220 10, 13 
12; 18:55:55 .2838 11, 13 
fugiens cens -3532 || ats. .....| .2285 || Olgas iel è 
A E .594 | aws.....- -478 | agi...... .642 


24. It will be noted that the values of the triads for the calculation of any 
one of the preceding group-factor coefficients are fairly constant, i.e., they 
satisfy (statistically) the conditions for one common factor. If the triads 
are written out for the calculation of ais,4 from all the tests of G4, it will be 
found that they vary from 


= Tig 147 19,14 = .001 x .035 in 
ligi) = cm 130 = .0003 


to 


_ Pis, 14716, 14 = .234 X .220 
715, 16 .151 


hs, 16 


= 3409, 
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with five triads nearly zero and the other fivesignificantly different. Clearly, 
the six variables of G4 do not measure a single factor, and the original pat- 
tern plan must be modified for these tests. 

25. By various procedures in 6.10, it is shown that the revised plan may 
be formulated as follows: Tests 14 to 17 involve a single factor, say B4, and 


TABLE C.11 
CALCULATION OF THE COEFFICIENTS OF By 


Test 14 Test 15 Test 16 Test 17 

jk tik jk tjk jk tik jk tjk 
ipit MB IE] Ae 1 15] 6s 
T5 v onde 3386 18,11. s 2587 15,165 5v; .1500 
dire umm. 2266 || alae] -4248 || ana) Say 


Tests 17 to 19 involve another factor, say Bs, so that Test 17 is of complexity 
3 while the other tests are still of complexity 2. The coefficients of B, are 
then calculated in Table C.11. Since there are just three tests that measure 


Bs, each of their coefficients is given by one triad. Thus 


— .231 X .137 m 
Qs = V/lis,19 = ae = V.2434 = .493, 


231 X -130 L \/ 9792 = .468, 


ais, = Vh E 137 
— 137 X .130 — h 
19,5 = Vlys,is sy  — v .0771 = .278. 


cation of the original pattern plan involves the sig- 
nificant general-factor residual 716,24 = .255 between two Lexar ae 
were not together in the initial grouping. A doublet is dn or the 
two tests, and its coefficients, according to (6.50), are taken to be 


955 — 117 = V.138 = 371. 


26. Another modifi 


dio > do41 = 
i blet coefficients, and the general- 
27. Th up-factor coefficients, the dou nts, 
factor be = now be entered in Table 6.8 to give the final factor 
t " . 
XT final residuals can now be obtained by means of the formula 


Fir = Ti — Gju ku + 
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If the variables z; and z; do not measure the same group factor, then their 
general-factor residual is the final residual. On the other hand, if variables 
j and k involve the same group factor, then the product of their group-factor 
coefficients must be subtracted from their general-factor residual to obtain 
the final residual. For example, Tests 1 and 5 are in different groups, and, 
therefore, their general-factor residual, — .022, is the final residual. Tests 
6 and 9, however, involve the same group factor, and their final residual is 
not .392 but 


Fes = .392 — .559(.628) = .041. 


Record the final residuals for the variables that involve a common factor, 
beyond the general, above the principal diagonal of Table C.6. 


TABLE C.12 
FREQUENCY DISTRIBUTION OF FINAL RESIDUALS 
Value of Residual Frequency Value of Residual Frequency 
3 —.130——. 8 
5 —,150——. 6 
6 —0—-.1B5l.ags [oes teo s 
8 QO 11er sudo etm 
12 —.210—-—.191....... |]... 
25 —.280— —.211....... |i ies 
33 —.250—-. 1 
30 
=. 39 E a aa 
= 2 276 
E 29 MB us aar anm - 
= 15 E. 
- H Standard deviation... . .0655 


29. According to the methods of Chapter VI, the factor pattern of Table 
6.8isa valid description of the twenty-four psychological variables, and the 
factorization has been carried to the proper stage as judged by the various 
statistical tests. The degree of factorization represented by this pattern is 
also in agreement with the more crude criteria for “when to stop factoring” 
which have hitherto been followed in factorial analyses, namely, that the 
communality of a variable should not exceed its reliability and that the 
standard deviation of the final residuals should be of just about the same 
magnitude as the standard error of a zero correlation in a sample of equal 
size. In Table 6.11 it is shown that the first of these standards is met, while 
Table C.12 gives the frequency distribution of the final residuals of Table 
C.6. Since the standard deviation (.0655) of the final residuals is less than 


| 
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the standard error (.0830) of a zero correlation, the second standard is 
also satisfied. - 

30. The variance of each variable which is not accounted for by the 
common factors can easily be deduced by the formulas of 2.3. In Table 6.11 
the communality and uniqueness of each variable is presented, and then 
the coefficients of the unique factors are obtained. These are given in Table 
6.8 along with the common-factor coefficients. Each row of this factor pat- 
tern then completely expresses a variable as a linear function of the factors. 


APPENDIX D 


OUTLINE FOR COMPUTATION OF A 
PRINCIPAL-FACTOR PATTERN 


A principal-factor pattern can be procured for any matrix of correlations 
by the iterative method of 7.4. The form of the numerical calculations is 
exhibited in the following steps, in which the method is applied to the ex- 
ample of eight physical variables introduced in 7.5. 

1. Determine the best possible estimates of the communalities for the 
given set of variables. This problem is considered in detail in 4.6 and in 7.2, 
and will not be repeated here. For the illustrative example, the choice of 
communalities is discussed in 7.5. 

2. Insert the estimates of the communalities in the principal diagonal of 
the correlation matrix and fill in the complete matrix. It has usually been 
suggested that only the lower-left half of a matrix of correlations be written, 
since such a matrix is symmetric, and the correlations of any variable with 
all the others can be determined from this portion. In the present method 
of analysis, however, it will be found more convenient to write the sym- 
metric matrix in full in order to facilitate the squaring of the matrix. The 
correlation matrix R for the given data is presented in Table D.1. 


TABLE D.1 
CORRELATION MATRIX: R 

1 2 3 4 5 6 7 8 Sj afp 
1 854 .846 .805 .859 .473 .398 .301 .382 4 00 
2 846 .897 .881 .826 .376 .326 .277 .415 4 E ; pen 
3 805 .881 .833 .801 .380 .319 .237 345 4.601 936 
4 859 .826 .801 .783 .436 .329 .327 .365 4.726 961 
5 473 .376 .380 .436 .870 .762 .730 629 4.656 947 
6 398 .326 .319 .329 .762 .687 .583 577 3.981 809 
7 301 .277 .237 .327 .730 .583 .521 539 3.515 715 
8 382 .415 .345 .305 .629 .577 .539 .579 3.831 779 


3. Obtain the sums of the rows (or columns 


) of R. For any variable j 
this sum is 


in which the estimate of the communality Àj is used for r;;. For the ex- 


ample, n = 8 and the sums are given in the column alongside the matrix 
of correlations in Table D.1. 
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4. To determine the first-factor coefficients, take as the set of trial values 
the numbers af}, which are the quotients of S; by the largest (absolute) 
sum. It should be noted that these numbers will not be used immediately 
in equations like (7.12), but it is convenient to refer to them as trial values. 
The numbers of? (j = 1, 2, . . . , 8) for the example, which are obtained by 
dividing the numbers S; by 4.918, are also given in Table D.1. It may be 
found more convenient, especially if the number of variables is great, to 
obtain the reciprocal of the largest S; and to multiply all the sums by it. 

5. Square the matrix of correlations. Any element, off the diagonal, of 
the resulting matrix R? is the sum of all the products of corresponding ele- 
ments in two columns of R. The diagonal elements are merely the sums of 
the squares of the elements in the corresponding columns of R. Thus, the 
element in the jth row and the kth column of R? is given by 


D roito (j, k = 1. AEE 


v=1 


The matrix R? for the given data is presented in Table D.2. The entry in 


TABLE D.2 " 
SQUARE OF CORRELATION MATRIX: R? 
1 2 3 4 5 6 d 8 sp sy a? 
870 22.370 1.0000 
1 || 3.450 3.450 3.301 3.325 2.577 2.185 1.903 2.179 || 22 
2 || 3.450 3.475 3.322 3.333 2.471 2.093 1.815 2.115 22.074 D 35 
3 || 3.301 3.322 3.181 3.188 2.341 1.983 1.718 2.003 21.037 2 don A 
4 || 3.325 3.333 3.188 3.213 2.461 2.084 1.810 2.085 21.499 din e 
5 || 2.577 2.471 2.341 2.461 2.966 2.550 2.278 2.372 || 20.016 20. 1s HR 
6 | 2/185 2.093 1.983 2.084 2.550 2.200 1.965 2.041 17.101 m ied 
7 || 1.903 1.815 1.718 1.810 2.278 1.965 1.765 1.820 15.07: dE nie 
8 || 2:179 2.115 2.003 2.085 2.372 2.041 1.820 1.925 16.540 16. s 


the second row and third column is given by 


8 
Mars = .846(.805) + .897(.881) + .881(.833) + .826(.801) 


g + .376(.380) + .326(.819) + .277(.237) + .415(.345) = 3.322, 
i i i imi ly the diagonal 

while any of the others is obtained similarly. of course, on. 

Vom ee ont the entries above (or below) the diagonal need to be a 

because the square of à symmetric matrix is also symmetric. The comple 

matrix is written, however, for convenience of further Mei AE 
6. A check is available on ihe squaring process. Compute the pr 


344 FACTOR ANALYSIS 


R by the column of values 5; of Table D.1. The result is the column of 
values Tí? of Table D.2. These values are given by the following formula: 


8 
TP = D rS, CF Be Bhi 


v=1 


so that the first entry, for example, is 


8 
TP = S r8, = .854(4.918) + .846(4.844) +... 4+.382(3.831) = 22.370. 


v=1 


Now, add the rows of R? and denote the respective sums by S(?. Corre- 
sponding values of Sf? and Tí? should agree except for errors of rounding. 

T. 'Take as the next set of trial values the numbers af), which are the 
quotients of Tj? by the largest check sum T(?. Every number T? is di- 
vided by 22.370 in Table D.2 in obtaining the trial values aff. 


TABLE D.3 


FOURTH POWER OF CORRELATION MATRIX: R‘ 

1 2 3 4 5 6 7 8 st qe ai? 
65.54 64.94 61.95 63.06 56.04 47.80 42.00 46.59 || 447.92 447.93 1.0000 
64.94 64.38 61.41 62.49 55.27 47.13 41.41 45.98 || 443.01 443.02  .9890 
61.95 61.41 58.59 59.61 52.67 44.91 39.45 43.82 | 422.41 422.43 .9431 
63.06 62.49 59.61 60.67 53.86 45.93 40.36 44.78 430.76 430.76 .9617 
56.04 55.27 52.67 53.86 50.40 43.06 37.97 41.61 390.88 390.90 .8727 
47.80 47.13 44.91 45.93 43.06 36.80 32.45 35.55 333.63 333.64 .7448 
42.00 41.41 39.45 40.36 37.97 32.45 28.62 31.33 


| 293.59 293.60 .6555 
46.59 45.98 43.82 44.78 41.61 35.55 31.33 34.39 || 324.05 324.05 7234 


CON Oo Pewee 


8. Compare the trial values af? with the corresponding af} of the first 
set. When there is no appreciable variation between two successive sets of 
trial values, then the squaring process has been carried far enough. This 


is the scheme for determining the number of times a matrix should be 
squared. In the present case the values af? differ 


sponding afP (see Table D.5) 
closed. The standard recomme 
sive trial values to within five 
final results. Here it is planne 
coefficients, the same as in t 
values of the discrepancies 

9. Square the matrix of c. 
the successive sets of trial 
various powers of R, alway 


sufficiently from the corre- 
so that the need for further squaring is dis- 
nded is to obtain agreement between succes- 
units in the last decimal place desired in the 
d to retain three decimal places for the factor 
he original correlations. Hence the absolute 
between trial values should be less than .005. 
orrelations a sufficient number of times to make 
values approximately equal. In calculating the 
s determine the check column T; first. Thus for 
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the eth power of R, the column of values T(? can be calculated from R*?and 
Sí*/? before the elements of R! are obtained. Then the values of aí) may 
be computed and if they agree with the values a{{/*), the calculation of the 
elements of R* is obviated. 

In the illustrative example, the values Tí? are obtained from R? and 
S$, and the af? are calculated. These values are not close enough to the 
corresponding af? and so the elements of R* are computed in Table D.3. 
Next, the values of TÍ? are determined in Table D.4. Since the maximum 


TABLE D.4 
EIGHTH POWER OF CORRELATION MATRIX: R5 
1 2 3 4 5 6 7 8 SP ze Dod 
176738 1.0000 


.. 174853 .9893 
. 166733 .9434 
.. 169980 .9618 
.. 153733 .8698 
. 131201 .7423 
. 115430 .6531 
127505 .7214 


difference between af} and aff is only three thousandths, this agreement is 
sufficiently close. It is therefore not necessary to caleulate the entries of 
R5, The differences between successive a determinations are summarized 


in Table D.5. 


TABLE D.5 
DIFFERENCES BETWEEN SUCCESSIVE 

TRIAL VALUES 

apos | alt af? 

0000 .0000 

0022 .0003 

0026 .0003 

0006 .0001 

—.0221 — .0029 

—.0197 — .0025 

— .0190 — .0024 

— .0160 — .0020 


10. Employ the last set of trial values as the arbitrary numbers aj; in 


the equation 


(7.12bis) aj, = om ; 
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that is, multiply the correlation matrix by the trial values to obtain the 
new set of numbers a/,. Then divide every number aj, by the largest one 
of these to get the next set of trial values. For simplicity of notation, again 
represent these new values by aj in the tabular arrangement. Multiply R 
by these new values to obtain the new aj, and aj and continue this process 
until successive values of aj are practically constant. When the squaring 
of R has been carried to the stage indicated in step 9, the values of aj: usual- 
ly will satisfy (7.13) immediately: 


(7.18%) ai, = Aan, 


that is, the values a/, will be directly proportional to the corresponding 
values aj. 


In Table D.6 the first column of arbitrary numbers aj are the values 


a{?), determined by the squaring process in step 9. These numbers are then 


TABLE D.6 
CALCULATION OF THE F, COEFFICIENTS 
Vari 1 — 
ae aj aj a aj maj V/ V Sa 
«lus 1.0000 4.4556 1 858 
2.. 9893 4.4083 9894 849 
3.. 9434 4.2038 9435 810 
4.. 9618 4.2852 9618 825 
5.. 8698 3.8757 8698 747 
6.. 7423 3.3076 7423 637 
7 6531 2.9099 6531 561 
8 7214 3.2142 7214 619 


8 
X = 4.4556, Jah = 6.0487 


V/V Zaj, = .85827 


multiplied by R to get aj,. For example, 
a, = .854(1.0000) + .846(.9893) + . . . + .382(.7214) = 4.4556. 


After all the values aj, have been obtained, they are divided by the largest 
of them (aj, = 4.4556) in the calculation of the new quantities aj. Since 
the maximum discrepancy between the old and the new values aj is only 
.0001, these numbers are accepted as stationary. i 
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11. The value of a/, corresponding to an = 1.0000 is the first characteris- 
tic root 41. Then the coefficients of the first factor can be calculated by 
means of 

aa Vs 
Qi = ER - — 
aj tai, +... +a) 


(7.7bi8) 


In the example, Xi = 4.4556 and the coefficients aj; are given in the last 


column of Table D.6. 
12. A check on the final determination of the aj is provided by the 


fact that 
n 
da =M. 
j=1 
the sum of the contributions of the first factor to the total 
communality must be equal to the first characteristic root. The value of 


X from the analysis is 4.4556 and the sum of the squares of the coefficients 
is 4.455, so that the check is satisfied (within rounding errors). 


In other words, 


TABLE D.7 
PRODUCT MATRIX: Qi=llanau ll 

1 2 3 4 5 6 7 8 Ej — anDi 

5.067 ` 5.067 

2 T8 To. 5.014 5.014 
3 695 .688 .656 . 4.783 4.784 
4 .708 .700 .668 BBI ass eee HOS 4.873 4.872 
5 “641 .634 .605 .616 .588 ...- oeo vn 4.411 4.412 
6 "547 .541 .5160 .526 .476 „406 .... ee 3.763 3.762 
7 “481 .476 .454 .463 .419 1357 .315 .... | 3.312 3.313 
8 '531 1526 .501 .511 .462 "394.347 .383 || 3.655 3.656 


be determined, the first-factor residu- 


13. Before additional factors can 
of the first-factor coefficients may be 


als are required. A table of products : 
prepared to facilitate the calculation of these residuals. The elements 


Diz (=1pr;) of the product matrix Qi = ||:piell = lana are obtained as 
follows: the values in the first column are the respective products of an 
by itself and every other a (j = 2,3, +++) n), the values in the diagonal 
and below it in the second column are the respective products of azı by it- 
self and all remaining coefficients qi (23,45.. n), and so on for the 
values in and below the diagonal in all the other columns. Since the product. 
matrix is symmetric, it is not necessary to write the values above the diago- 
nal. The matrix Q; for the given data is presented in Table D.7. 
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To assist the reader, a summary of the notation referring to the various 
matrices employed in the principal-factor method is presented in Table D.8. 


TABLE D.8 
NOTATION 
Original First- First- Second- Second- 
Correla- Factor Factor Factor Factor 
tions Products | Residuals | Products | Residuals 
Elerent. rics soosi ovest ou Tjk ipik Wik 2Dik Tjk 
MRUEDOL asie iwak ee oon R Qi R: Q: R: 
Sum of elements in row j of 
i111) 5,» REOR Sj Ej, Sa Ej Sg 
The eth power of matrix. . R* x Ri H £j 
Sum of elements in row j of 
eth power of matrix..... Sp Eg d Eg SY 


14. To check the calculation of the elements of the product matrix Qi, 
first obtain the sums of the complete rows (or columns), that is, 


n 
En = X anan (m3, 9 soy 8), 
k=1 


and compare with the corresponding values of a; Dı, where 


n 


D; = San. 


k=1 


The sum of the first-factor coefficients for the giyen data is D, = 5.906. 


Then the check, say, for the elements of the third row and column is given 
by the agreement of 


En = .695 + .688 + .656 + .668 + .605 + .516 + .454 + .501 = 4.783 
and 


asıDı = .810(5.906) = 4.784 , 


and the remaining elements of Q; are similarly checked. The values Ej, and 
aj,D, are also recorded in Table D.7. 


15. Subtract the values in Qi from the corresponding entries in R to get 
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the matrix of first-factor residuals Ri. This matrix is presented in Table 
D.9, and is written out in full to simplify later multiplications with it. The 
sums of the rows, Sj, are given in a column alongside the matrix Ri. These 


TABLE D.9 
MATRIX OF FinsT-FACTOR RESIDUALS: Ri 
1 2 3 4 5 6 7 8 Si aj? 
1| “WS .118 .110 .151 —.168 —.149 —.180 —.149 | —.149 — .6082 
2 .118 .176 .193 .126 —.258 —.215 —.199 —.111 —.170 — .6939 
3 .110 .193 "177 -133 —.225 —.197 —.217 —.156 | —.182 — .7429 
4 -151 .126 .133  .102 —.180 —.197 —.136 —.146 | —.147 — .6000 
5| —.168 —.258 —.225 —.180  .312 .986  .311 .167 .245 1.0000 
6| —.149 —.215 —.197 —.197  .286 .281 .226  .183 .218 . 8898 
7| —:180 —.199 —.217 —.136 .311 .226 -206 :192 || -203  .8286 
8|| —.149 —.111 —.156 —.146  .167 .183  .192 .196 | .176 .7184 


sums should be equal to the differences between the sums of the respective 


rows of R and Q;, that is, 
Sa = S; — En. 


of trial values for the calculation of the second- 
r of Ri should be employed. It is not 
d squarings on R; to get this power, 
of the formula 


16. Toobtain the best set 
factor coefficients, an appropriate powe: 
necessary, however, to perform repeate 
because it can be obtained more simply by means 
(7.21bi8) Rp-R-AXUQ. 
Furthermore, since the actual entries of Rj, or any higher power of Ru, are 
not required for the determination of trial values if the sums of the rows 
are known, additional labor can be saved by means of formula (7.21). The 
values Sí? and Ej may be considered as elements of the matrices R? and 
Qi, respectively. Then, according to (7.21), 


SP = Si) — NEn, 


s in row j of the matrix R?, can be ob- 


dual entries in Ri. 
h block contains the derivation of the 
nted by the superscript on ajz. In 
copied from Tables D.1 and D-7, 
en the trial values af? are 
by the largest one of 


so that S{?, the sum of the element 
tained without calculating the indivi 

Construct Table D.10, in which eac 
trial values from the power of Ri represe 
the first block the values of S; and Ej are 
and the sums S; are obtained by subtraction. Th 
caleulated by dividing the corresponding sums Sj 
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them (in absolute value), that is, by Sa = .245. Record the values of Sí? 
from Table D.2 in the second block of Table D.10, retaining only two deci- 
mal places since all the work is based upon three significant figures, and 


TABLE D.10 


DETERMINATION OF TRIAL VALUES FOR THE 
CALCULATION OF THE F, COEFFICIENTS 


Vari- 
able Sj Ej Sj a sp MEQ si? a? 
j 
4.918 | 5.067 | —.149 | — .608 22.37 | 22.58| —.21| — .57 
4.844 | 5.014 | —.170 | — .694 22.07 | 22.34 —.27 | — .73 
4.601 | 4.783 | —.182 | — .743 21.04 | 21.31 —.27| — .78 
4.726 | 4.873 | —.147 | — .600 21.50 | 21.71 —.21 | — .57 
4.656 | 4.411 .245 1.000 20.02 | 19.65 .97 1.00 
3.981 | 3.763 .218 .890 17.10 | 16.77 .38 .89 
3.515 | 3.312 -203 |. .829 15.07 | 14.76 .3l .84 
3.831 | 3.655 .176 .718 16.54 | 16.29 .25 .68 
TABLE D.11 
CALCULATION OF THE F, COEFFICIENTS 
Vari- 
able En ajs 52 aja aja aja aja jim V Ai/ V Eas 
j 
lo — .57,— .865|— .580|— .8856|— .5851|— .8852|— .5851 —.328 
VPN — .73|—1.100|— .737|—1.1152,— .7368|—1.1148|— .7369 —.414 
Bie us — .73|—1.097|— .735|—1.1118|— .7345|—1.1112,| — .7345 —.412 
- — .902|— .605|— .9129.— .6031|— .9129|— .6034 —.339 
1.492) 1.000| 1.5136) 1.0000| 1.5129| 1.0000 .561 
1.348|  .903| 1.3666|  .9029| 1.3660}  .9029 .507 
1.300 .871| 1.3144 .8684| 1.3142 .8687| .488 
.987|  .662| 1.0005 .6610| 1.0001 .6610 .371 


8 
X-1.5129, J a}=4.8027 
m 


V/X/ V Eai = . 56126 


one additional figure is sufficient to assure the accuracy of the three figures. 
Compute the products Ej, with the value A, = 4.4556 taken from Table 
D.6, and again keep four figures. The sums S$? are obtained simply by sub- 
traction, and the corresponding af?) are then determined. These values are 
truly significant to only one decimal place, and in the one significant figure 
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they agree with a{}). If the calculations in the next block were attempted, it 
would be evident that corresponding values of Sf? and ME;, are edual to 
three significant figures. For example, S(9 = 448 and fH, = 448. Hence 
the sums S? (that is, Sf}? — XE;;) are insignificant and a(? cannot be ob- 
tained. It therefore follows that the best set of trial values for the calcula- 
tion of aj; is af}. 

17. Employ the numbers a, 
to multiply the matrix Ri to o 
values, for example, is given by 


al, = .118(—.57) + -118(—.73) + .110(—.73) +. - . —.149(.68) = —.865. 


(2) as the first set of trial values aj; by which 
btain aż, in Table D.11. The first of these 


Divide the values a/, by the largest of them (al, = 1.492) to obtain the 
next set of trial values, which are also designated by aj» for simplicity. 
Multiply Ri by these new values and divide the resulting ajz by the largest 

t of trial values. Continue this process 


one of them to obtain the next se 
until corresponding trial values in successive sets agree to three significant 


figures. In the computations, after the first or second iterations, keep four 
figures if three significant figures are desired for the factor coefficients. 
Three iterations of the form (7.12) were sufficient, in the present example, 


for stability in the trial values aj». 
18. The value aj, = 1.5129 corresponding to a5» — 1.0000 is the char- 
acteristic root s. Then the coefficients of the second factor can be computed 


by means of a formula like (7.7), as follows: 


ajo = ag VN (212,...,7)- 
Mal, + od, +... + one 


The coefficients aj; are given in Table DAL 
19. The final calculation of the coefficients aj» 


of the formula 


can be checked by means 


n 
Sah = Ae . 
j=l 

In the example, the sum of the squares of the eight coefficients 18 1.511 and 

As = 1.5129, so that the coefficients check. . 

20. To obtain additional factors proceed with the form of calculation as 
indicated in steps 13-19. The notation of Table D.8 will aid in the applica- 
tion of the method, for then the only changes in the preceding steps will be 


the advancement of the indices. 


352 


FACTOR ANALYSIS 


For the given data it will be shown now that no further factors are re- 
quired. The products of the second-factor coefficients sp;; = aj2ax2 given 
in Table D.12, are subtracted from the corresponding residuals ir;, of Table 


BNO ot 


"184 —:232 —.231 -—.190 315 


TABLE D.12 
PRODUCT MATRIX: Q2=|lanax]l 


—:166 —.210 —.209 —.172 .284 1257 |... 
—:160 —.202 —.201 —.165 .274 247 (238 .. 
—:192 —.154 —.153 —.126 .208 .188 .181 .138 


1 2 3 4 5 6 7 8 
NOB orare dew dert Tu Em 
136 SA arenen sorte y diste Ai did 
135 a DAA AIO) canas an ie 
111 140 .140 115 BEV cud 


| 
| 


D.9 to obtain the matrix of second-factor residuals which is presented in 
Table D.13. These residuals are obviously insignificant and so may be con- 
sidered as final. Inasmuch as the problem of factor analysis is to account for 


00-100 RO 


TABLE D.13 
MATRIX OF SECOND-FACTOR RESIDUALS: R, 


—.000  .OM 1058 | 


the total communality variance, a more definite check on the adequacy of a 
solution is afforded by the extent to which the sum of the contributions of 
the factors agrees with the original total communality. In the present ex- 
ample, two common factors account for practically 100 per cent of the com- 
munality. The percentage contributions of the individual factors are pre- 
sented in Table 7.4, where the complete principal-factor pattern is exhibited. 


APPENDIX E 
OUTLINE FOR COMPUTATION OF A CENTROID PATTERN 


By the method developed in 8.2, a centroid pattern can be obtained for 
any correlation matrix. The details of the numerical calculations are de- 
scribed in the following steps. A centroid solution for the first thirteen of 
the twenty-four psychological tests of 2.8 is obtained to illustrate the steps 
in the outline. 

1. Determine the best possible estimates of the communalities for the 
given data. The problem of estimation of communalities is treated in 7.2. 
The choice of communalities for the illustrative example is indicated in 8.3. 

2. Put the estimates of the communalities in the principal diagonal of 
the correlation matrix, writing the correlations in the lower-left half of the 
table only. The intercorrelations of the thirteen psychological tests are 
given in Table E.1, in which the communalities are also included. 

3. The procedure for the calculation of the coefficients of the first cen- 
troid factor is presented in this and the next three steps for the case of a 
matrix involving very few negative correlations. If a matrix contains & 
large number of negative correlations, then the procedure of steps 11-27 
should be followed. The present example is typical in that it involves a 
matrix of generally positive correlations. 

Obtain the sums of the complete rows (or columns) of the correlation 
matrix, Although only half of the intercorrelations are written explicitly, 
the complete sums can be obtained very simply. Thus, for any variable J, 
add the entries in row j to the diagonal and the entries in column j (includ- 
ing the diagonal value). This sum is represented by 


Si DT 


k 


of k, and r;; is the estimate 
dix the symbol Z will be 
eindexisinvolved; and, 
mation 


xtends over the n values 
Throughout this appen 


from 1 ton when à singl 
ved, the index for which the sum 


S, for the given data, are presented in 


where the summation e 
of the communality Aj. 
used to connote summation 
when more than one index is invo. 
is made will be specified. The sums 


Table E.1. 
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TABLE E.1 


CALCULATION OF THE C; COEFFICIENTS FROM THE CORRELATION MATRIX R 


Variable 


Check 


:314| 145] 140 

.489| .239| 321| 1327 
.592| 2.687| 3.168| 3.618 
.607| .355|  .418| .478 


.729 


.707 


.945| .395 
5.455| 5.340 
.721| .705) 


.698 


.408 


:535 


=57.314 


SOL ess 
-512| .599| 
3.690| 5.103 
.487| .674 


T =57.314 
VT = 7.5706 
1/VT - .1321 
D,- 7.871 
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4. Calculate the total T' of all the entries in the correlation matrix. This 
may be done simply by use of the formula 


T= > je 
For the example, 


13 


T= > 5 = 4.592 + 2.687 +... + 5.103 = 57.314, 
j=1 


and V/T = 7.5706, 1/VT = .1321. The sum T may then be checked by 


means of the formula 


Fag 5 Tik — UT, 


isk 


where all indices extend over the range 1 to n, and the first term indicates 
summation for both j and k under the restriction j € k. This check is also 
given in Table E.1. 

5. The coefficients of the first centroid factor now can be computed by 
means of formula (8.6), namely, 


Gun = —— (j21,23,...,0). 


The factor coefficient for any variable z; thus is obtained by dividing the 
sum of the n correlations of this variable with all others (including the com- 
munality) by the square root of the sum of all the correlations in the com- 
plete matrix. For a large set of variables, it is convenient to calculate the 
reciprocal of VT and to multiply all the values S; by it. The factor coeffi- 
cients for the given data are obtained by multiplying the respective sums 
S; by 1/VT = .1321 and are presented in the last row of Table E.1. 
6. A check on the preceding computation is available. Since 


S; 
aj = zm 
it follows that 
Di = zan = Se AS = et = VT. 


In the example, 
D, = Zaj = 7.571 " 


which agrees, except for errors due to rounding, with the value of YT. 
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7. Before additional factors can be determined, the first-factor residuals 
are required. A table of products of the first-factor coefficients may be pre- 
pared to facilitate the caleulation of these residuals, just as in the case of 
step 13 of Appendix D. The elements pix (—ipi;) of the product matrix Qi 
are obtained as follows: the values in the first column are the respective 
products of ay by itself and every other aj: (j = 2,3,... , n), the diagonal 
value and all the entries below it in the second column are the respective 
products of a; by itself and all remaining coefficients Ga (j = 3,4,... T 
and so on for the values in and below the diagonal in all the other columns. 
Since the product matrix is symmetric, it is not necessary to write the values 
above the principal diagonal. The matrix Qi = |ipal| = llanarl| for the 
given data is presented in Table E.2. For example, the elements in the 
first column of the product matrix are 


Pu =a}, = (607)? = 368, 
ipi? = Quaza = (-607)(.355) I5 à 
1Pis = anası = (.607)(.418) = .254 , 
ipii = Qna (.607) (.478) = .290, etc. 


8. To check the calculations of the products, step 14 of Appendix D can 
be applied. Employing the same notation as in the principal-factor method, 
the sums of the complete rows (or columns) of the product matrix Q, are 
designated by 


Ly = Danan (G = 52... 27), 
D 
or 
En = aD, 


which serves as a check on the products of the first-factor coefficients. For 
the given data the thirteen values of Ej and the corresponding checking 
values aD; are presented in Table E.2. 

9. Subtraet the values in the product matrix Q; from the corresponding 
entries in the correlation matrix R to obtain the matrix of first-factor residu- 
als Ri = [irl = [ri — ipis. The first-factor residuals are presented in 
Table E.3. For example, 

yu =ru — pn —.558 — 368 = — 199, 
1729 = Tag — Pa = .195 — .248 = — .053 , 


and all the other residuals, which are given in and below 


t the diagonal in 
Table E.3, can be obtained in a similar manner. 


468 


TABLE 1.2 


PRODUCT MATRIX: Qi- [asa] 


Variable 


.424 | .248 | .292 509 | 
.276 | .162 | .190 .332 MEN 
.326 | .191 | .221 .391 .288 
.206 | .173 | 204 -355 -262 
.400 | .239 | .282 -491 | 362 
4.596 | 2.688 | 3.166 | 3.619 | 5.519 | 5.353 | 5.459 : 5.337 | 5.285 
2.688 | 3.165 | 3.619 | 5.519 | 5.353 5.459 | 5.338 | 5.285 | 3.445 | 4.006 


4:996 | 


3.446 | 1.066 


TABLE E.3 
FiRST-FACTOR RESIDUALS: irj 


Variable 


1 


898 


12 


13 


- —.017| —. 
—.184| —.099| —. x 
—.096| —.093| . 1 
—.098} —.053| —. -. à 
—.160| —.105| —. -. - - - ; 
—.018} —.041| -. -. -. -. - ; ; 

018| —.028| —.064| —.073| —.140| —.249| —.170| —.072| —.227|  .363| .160| .294 |...... 

.080| 000) .039| .005| —.147| —.168| —.141| —.080| —.190|  .101| .173| .184 | .145 
—.004| —.001| .002! —.001 000| —.003| —.004|  .003 000} —.003| —.002| .002 | .001 

>» - om 
——————T T" ——— "P 
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TABLE E.3' 
SIGN CHANGES 


Variable 


UE did d 


"af 


09€ 


TABLE E.3" 
CALCULATION OF THE C; COEFFICIENTS 


Vari- 
able x 2 3 4 5 6 7 8 9 —10 =j -12 =13 Check 
o. .080 
2 — [000 
de — .039 
e = i005! T:=29 ik — Eri 
5.. -147| iix 
6. | 
27 168" T, =2(8.028) 
os 080, —2.318 
cel 190) 
YO. cris don 713.738 
1i. . 173) 
12.; E 
13. 145) 
Sa....| .224| .141| .580| .309| .954 1.311) 1.360| .729| 1.516, 1.785) 1.446| 2.048} 1.365) 7, = 13.738 
ejSa...|—.224| .141| 1550} .309| .954| 1.311] 1.360) .729| 1.516|—1.785|—1.446,—2.048 —1.305 ^ Jg. 
à T,- 3.7065 
1/VT,= .2698 
4m... —.080 088 .148| .083) .257| 35a) .867| .I9T.  .409— .482— .390— 553) — 868. D.- .000 
S m. -————— — Á 9 


Ape 
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10. A check on the subtractions involved in obtaining the first-factor 

residuals is afforded by 
Y re =S;- Eg. 
E 
The sum of all the residuals for each of the variables appear in Table E.3 
and may be checked with the differences S; — Ej. These numbers always 
must check exactly, of course, because no statistical discrepancies, such as 
rounding of numbers, are involved. The sum of the residuals for variable 5, 
for example, is 
Dirge = —.122 + .026 +... —-147 = .000 , 


while S; — Ej = 5.519 — 5.519 = .000. © 

The preceding check may be obviated, for in the residual-factor space 
the centroid is at the origin, and hence it is expected that each of the residual 
sums will be approximately zero. Then the foregoing check may be em- 
ployed in individual cases where the difference of a particular sum from 
zero is questionable. Thus, if it is doubted that the largest sum (in absolute 
value) differs from zero only because of rounding errors, the check may be 
applied to it. In the example one of the largest sums is Zjr;, = —.004, and 
S; — En = 5.455 — 5.459 = —.004. 

11. To remove the centroid from the origin in the residual-factor space 
and to increase the contribution of the second factor to the residual vari- 
ance, certain variables are reflected in the origin. The variables to be re- 
flected are determined from a table similar to Table E.4, in which the num- 
ber of negative residuals for each variable is recorded and minimized by 
the reflection of variables. In this table put the numbers of the variables 
in the stub and allow two rows at the bottom of the table for the totals in 
the respective columns and the differences between successive totals, which 
will be useful for checking purposes. The first column* of the table should 
be headed “Reflected Variable," and a minus sign will be placed in this 
column opposite any variable which is to be reflected. The remaining col- 
umns are introduced successively as an increasing number of variables are 
reflected. 

12. Count the number of negative signs for each variable in the residual 
matrix of Table E.3 and record in the second column and in the appropriate 
row of Table E.4. The second column of Table E.4 is headed “Before Re- 
flection,” because the values put in this column refer to the number of nega- 
tive residuals before any variables are reflected. It should be noted that, 

* Here, and in the remainder of this outline, the column numbers refer to the body 
of the table, excluding the stub. 
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although only half of the symmetric residual matrix is written explicitly, 
the number of negative signs to be considered for each variable is that of 
the total matrix. In other words, when counting the number of negative 
signs read across the row and down the column for a specified variable. For 
example, the number of negative signs, before any reflections, for variable 
lis 7. 

13. Add all the entries in the second column of Table E.4 and record the 
sum in the “Total” row. In the present example this sum is 96. The count 


TABLE E.4 


NUMBER OF MINUS SIGNS FOR FIRST-FACTOR RESIDUALS AFTER 
SUCCESSIVE REFLECTIONS OF VARIABLES 


Re TT ArrER REFLECTION oF Successive VARIABLES 
VARIABLE FLECTED REFLEC- 

VARIADLE TION 10 11 12 13 í 

7 6 5 6 7 5 

y 6 5 4 5 6 

7 6 5 4 5 6 

7 6 5 4 5 6 

7 6 5 4 3 2 

8 7 6 5 4 3 

8 7 6 5 4 3 

6 5 4 3 2 1 

8 7 6 5 4 3 

9 3 2 1 0 1 

9 10 2 1 0 1 

8 9 10 2 1 0 

5 6 7 8 4 3 

s 96 84 68 52 44 40 

Difference. ...|.... 12 16 16 8 4 


of negative signs may be checked by re-counting the negative residuals in 
'Table E.3, which should be equal to one-half of the total recorded in the 
second column of Table E.4. 

14. Pick the variable with the largest number of negative signs to be 
reflected first. If several variables have the same maximum number of 
negative signs, any one of them may be arbitrarily selected for reflection, 
In the example, variables 10 and 11 have a maximum of 
and zio is arbitrarily selected for reflection. 

15. Opposite variable 10 in the first column of Table E.4 place a minus 
sign to indicate that this variable is to be reflected. An adjustment in the 
number of negative signs for each variable will be made as if variable 10 
were reflected in Table E.3, (i.e., as if all the signs for variable 10 were 


9 negative signs, 


oS 
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changed) and these results will be recorded in the third column of Table E.4. 
This column is headed “10,” to indicate that the count of negative signs for 
each variable is that after variable 10 is reflected. 

16. For the variable being reflected the adjusted number of negative 
signs is (n — 1) minus the number of negative signs it had before reflection. 
Upon reflection of a given variable, every residual which was positive be- 
comes negative and every negative residual becomes positive, except that 
the value in the diagonal of the residual matrix remains unchanged. The 
value (n — 1) is the total number of entries in a given row and column of 
(the lower half of) Table E.3, ignoring the diagonal In the example, 
n — 1 — 12 and the entry for variable 10 in the third column of Table E.4 
is12—9-3. 

17. It is not necessary to change all the signs of the residuals for the 
variable being reflected in order to count the number of negative signs for 
the other variables after the reflection. Instead, consider the sign of each 
entry except the diagonal in the row and column of Table E.3 for the vari- 
able being reflected, aad proceed as follows. 

a) If the entry for a particular variable, which was not previously re- 
flected, is positive, increase by one the number of negative signs for that 
variable recorded in the second column of Table E.4 and record the new 
value for that variable in the third column of Table E.4. For example, the 
entry for variable 11 in column 10 of Table E.3 is positive, and since zii was 
not previously reflected, the number of negative signs for it is increased one, 
from 9 to 10, in Table E.4 after variable 10 is reflected. 

b) If the entry for a particular variable, which was not previously re- 
flected, is negative, decrease by one the number of negative signs for that 
variable and record the new value for that variable in the third column of 
Table E.4. For example, the entry for variable 1 in row 10 of Table E.3 is 
negative, and since z; has not been reflected, the number of negative signs 
for it is decreased one, and the result 6 is recorded for it in the third column 
of Table E.4. 

The general rules for sign changes are formulated conveniently in 
Table E.5. 

18. The count of negative signs after variable 10 has been reflected can 
be checked by the following procedure. Add all the entries in the third col- 
umn of Table E.4, getting 84; and subtract this total from the preceding 
one, obtaining 12 for the difference. This difference should be twice the 
difference between the number of negative signs for variable 10 before and 
after reflection. In the example, 


12 = 2(9 — 3), 
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and so the arithmetical work involved in the third column of Table E.4 is 
checked. 

19. Pick the variable having the largest number of negative signs in the 
third column of Table E.4 as the next one to be reflected. This is variable 11, 
which has 10 negative signs. Put a minus sign opposite variable 11 in the 
first column and the number “11” at the head of the fourth column of 
Table E.4 to indicate that this variable is being reflected. Then adjust the 
number of negative signs for cach variable as if variable 11 were reflected 
in Table E.3, and record these results in the fourth column of Table E.4. 


The adjustments are made according to the procedure outlined in steps 
16 and 17. 


TABLE E.5 
RULES FOR SIGN-CHANGE ADJUSTMENTS 
Entry in Row (or Entry in Row (or 
Column) of Re- Column) of Re- 
flected Variable flected Variable 
Is Positive Is Negative 
Not previously reflected (or re- 
flected an even number of Increase one Decrease one 
times) 
Previously reflected once (or any Decrease one Increase one 
odd number of times) 


For example, the entry for variable 1 in row 11 of Table E.3 is negative 
(second column of Table E.5), and, since z, has not been reflected (first row 
of Table E.5), the rule for the sign adjustment is given in the upper-right 
cell of Table E.5. The number of negative signs for z, is reduced from 6 to 
5, which is recorded in the fourth column of Table E.4. Another example, 
that of variable 10, may be considered. The entry for variable 10 in row 11 
of Table E.3 is positive, and, since ziy was previously reflected once, the 
rule for the sign adjustment is given in the lower-left cell of Table E.5. The 
number of negative signs for variable 10 is then changed from three to two. 

Check the values in the fourth column of Table E.4 by the method indi- 
cated in step 18. 

20. If zero values should appear in any of the correlation or residual 
tables, they may be treated as positive numbers in making sign adjustments 
for the reflection of variables. The diagonal values of the residual tables are 
not considered in the count of negative signs, for, if a variable is reflected 
its “‘self-correlation” remains unchanged. f 

21. It may happen that a variable which had already been reflected may 
again appear as the variable with a maximum number of negative signs 
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after several other variables are reflected. In this case the variable is re- 
flected again, changing the minus to plus in the first column of Table E.4, 
and the number of minus signs is adjusted for each of the variables. 

22. The reflection of variables is continued until each variable has less 
than n/2 negative residuals. In the example n — 13, so that the reflections 
are carried to the point where there are six or fewer negative signs for each 
variable. It will be noted that upon reflection of variable 1, the entries in 
the last column of Table E.4 are six or less. 

23. The variables having minus signs in the first column of Table E.4 
now may be actually reflected in Table E.3. In order to make the procedure 
perfectly clear, two additional tables, E.3' and E.3”, are included. In prac- 
tice, however, these additional tables may be obviated by incorporating 
them in Table E.3. That procedure is indicated by Table E.7, in which the 
third-factor coefficients are calculated. 

24. First place a minus sign before the column number of each variable 
which is to be reflected, i.e., before variables 1, 10, 11, 12, and 13 in Table 
E.3'. This is done for the convenience of the subsequent sign changes in 
this table. 

The signs of the residuals irj, may be changed according to the following 
formula: 

Rik = cjeli ik) . 


It will be recalled that the epsilons are merely algebraic symbols for the 
if neither z; nor z, was reflected, or if both 


plus or minus signs. Hence, 
variables were reflected, then Tijk = jk’ but, if only one or the other of 2; 


and z, was reflected, then rijs = —irji. First go through the upper half of 
Table E.3, one row at a time, and insert minus signs according to the above 
rules. A convenient procedure is to look at each entry of the first column, 
note the adjusted sign, and when this sign should be minus, record a minus 
sign in the corresponding cell of the first row of the upper half of the table. 
Check the total number of minus signs for the first variable with the number 
given for that variable in the last column of Table E.4. Then proceed to 
the second column of the lower half of the table, note the sign changes, and 
record the minus signs in the second row of the upper half of the table. The 
count of six minus signs for variable 2, in the second column and second row 
of the upper half of Table E.3', agrees with that given in the last column 
of Table E.4. Continue this process of sign changes for every variable in 
Table E.3/. As an additional check, the total number of minus signs in the 
upper half of Table E.3' must be equal to one-half of the total given in the 


last column of Table E.4. 
25. Now, merely copy the values (without any algebraic signs) from the 
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columns of the lower half of Table E.3 into the corresponding rows of the 
upper half of Table E.3. This is done in Table E.3” for the example. The 
values so obtained are the residuals of the reflected variables. 

26. The second-factor coefficients are caleulated from the values in the 
upper half of the table.* The sum of the residuals (after reflection) for 


any variable z; is 
Sn = » Tiik; 
k 


and the total of all the reflected residuals is 
Tı = 2p. 


The thirteen sums Sj; and the total T, are presented in Table E.3^". The 
square root and the reciprocal of the square root of Ty are also given. The 
algebraic signs of the sums S; are changed for the variables that have been 
reflected, and the resulting values, 1S; = «Sj, are presented in Table E.3". 
Finally, by multiplying each 1S; by 1/ vmm; the respective coefficients ajz are 
obtained. These are the coefficients for the observed variables, not the re- 
flected ones. 

27. A useful check on the calculation of the second-factor coefficients 
(and the coefficients of all other factors except the first) is given by the fact 
that the sum of the coefficients must be approximately zero if a centroid 
system has been obtained. In the example, 


Ds = Zaàjs = .000. 


Another check, like the one given in step 6 for the first factor, follows from 
the relation 


ZS5 Ti 
Sedae E XE a 
€j0j2 VT, VT, T5 ë 
The sum of the factor coefficients for the reflected variables must be equal 
to the square root of the total of all the reflected residuals. The coefficients 
for the reflected variables are merely those given in the last row of Table 


E.3” with the signs changed for all variables which have been reflected. 
Thus, 


Zejaj; = .060 + .038 + .148 +... + .368 = 3.706 , 
which agr 2s with the value of Yn. 


* It may be noted again that this portion of the table was separated out as Table E.3” 
for clarification, but in practice only one Table E.3 need be made. 


—-——9— 4 


` - 
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28. Proceed, asin steps 7 through 27, to determine the third-factor coeffi- 
cients. The calculations leading to a;; are given in Tables E.6, E.7, and E.8. 
This procedure is repeated over and over again until sufficient factors are 
obtained, which account for the total communality. 

It is evident from the third-factor residuals, given in Table E.10, that 
three factors are sufficient to account for the original correlations. Further- 
more, the sum of the contributions of the three factors agrees well with the 
original total communality. This is indicated in Table 8.1, where the cen- 
troid pattern for the thirteen psychological tests is presented. The percent- 
age contribution of each factor to the total communality is also given in 
Table 8.1. The discrepancy between the original and calculated commu- 
nality for each test is given in the last column of Table 8.1. The sum of the 
thirteen differences is —.030, or —0.4 per cent of the original total commu- 
nality (6.936). This indicates a very good factorization, with a slight tend- 


ency for overfactoring. 


89g 


TABLE E.6 
PRODUCT MATRIX: Q:=|]ajaxl| 


Variable 


LAP Et Sg 
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TABLE E.7 
SECOND-FACTOR RESIDUALS AND THE CALCULATION OF THE C, COEFFICIENTS 
Variable -1 -2 -3 | -4 5 6 1 8 | 9 10 11 i2 | eg Check 
dee: .186| .105|  .158| .183| .107| .073| .112 .084  .073| .189| .041| .015| .058| 
Bev .105| .076 .163| .057,—.016| .030| .113| .100, .069 -087| .026| .007| .014 
3.. 158| .163 165| .093| .096| .080| .132|—.058| .169 .194| .075|—.018| .093 
4.. .183, .057 .093. -079) .142| .040, .040,—.038| .043| .078| .115| .027| .036| T, 225 n4 -Zrui 
5.. — .107| .016|— .096|—.142| .049| .016| .036| .013  .109 .103| .053| .002| .052 mn 
6.. — .073!— .030) — 080! — 040) 016} .016| .082|—.041| .076 -052! .111|—.053| .038| 
[d — .112,—.113,— .132 —.040  .036| .082| .095| .039 .032|  .095 —.012| .033| .006, 7.—2(5.334)—.927 
8.. — .084 —.100 .058| .038| .013,.—.041| .039| .035,—.041 .059,—.002| .037 -008| 
Dies — .073|—.069| — 169, — .043 109  .076 032| — .041 104 049 065, — .001 039 =9.741 
a m —.087|— .194/|—.078| .103| .052 095| .059, .049 115| .052 096 076) 
= —.026,— .075,—.115 053| .111|—.012| — .002! 065 052, .009|—.050|—.029 
a — .007 018,—.027| .002; — .053 033| .037|—.001 096| — . 050| — .012 020, 
014 093, .036, — .052) — .038| — .006| — .008| —.039| — .076 029| — .020. 010 
—.001 d 001| .000 —.002,—.003| .003  .000|— .003,—.002) .001| .001 T,29.741 
.831| 1.342! .895| .762| .520, .803| .195| .786 1.245| .454| .103| .421 VT, 3.1211 
—.881|—1.342 —.895| .762, .520, .803| .195| .786| 1.245 .454| .103,—.421 1/VT,= .3904 
—.266|— .429|—.287, .244| .167| .257| .062, .252|  .399| .145| .033/—.135 D= —.001 


m 


TABLE E.8 


NUMBER OF MINUS SIGNS FOR SECOND-FACTOR RESIDUALS 
AFTER SUCCESSIVE REFLECTIONS OF VARIABLES 


AFTER REFLECTION oF Suc- 


Re- BEFORE CESSIVE VARIABLES 
VARIABLE FLECTED Re- 
VARIABLE | FLECTION 
1 2 4 13 3 
8 4 3 2 1 0 
7 8 4 3 2 1 
6 7 8 9 10 2 
7 8 9 3 2 1 
4 3 4 3. 2 i 
7 6 5 4 3 2 
6 5 4 3 2 1 
6 5 4 5 4 5 
7 6 5 4 3 2 
5 4 3 2 1 0 
7 6 5 4 5 4 
7 6 5 4 3 4 
7 8 9 10 2 1 
Total. ...|. 76 68 56 40 24 
Difference. . 8 8 12 16 16 
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TABLE E.9 
PRODUCT MATRIX: Q;- llajsazl| 


Variable 


148 
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TABLE E.10 
THIRD-FACTOR RESIDUALS (FINAL RESIDUALS) 


Variable 


APPENDIX F 


EVALUATION OF THE ROOTS OF A 
POLYNOMIAL EQUATION 


In obtaining a derived principal-factor pattern, the solution of a poly- 
nomial equation of the third, fourth, or higher degree is required. Several 
methods are available for computing the roots of such an equation to any 
desired number of decimal places. Each method, however, requires some 
preliminary information concerning the root to be calculated. It is neces- 
sary to know the limits between which the root lies and that there is no 
other root between the same limits. Thus it would be sufficient to know 
that the root is between 5 and 6, provided that there is no other root be- 
tween these limits. If there were another root within this range, narrower 
limits would be necessary. When limits are obtained such that only one 
root is contained in the interval, then the root is said to be isolated. 

As the first step toward the isolation of the roots of an equation of the 


mth degree, 

(E1) f(a) = um aw"! + am? + es 4+ dma + am = 0, 
presented. The term “variation of sign" 
terms of a polynomial or equation 
ariations of sign of an equation 
ive terms (when written 


Descartes’s rule of signs will be 
is used to indicate that two consecutive 
have coefficients of unlike signs; and by the v 
is meant all the variations presented by consecut: 
in order of decreasing powers of the variable). 
DzscanrEs's Rute. The number of positive real roots of an equation f(u) = 


0 is either equal to the number of its variations of sign or is less than that number 


by a positive even integer. The number of negative real roots of f(u) = 0 is 
either equal to the number of variations of sign of f(—4) = 0 or is less than 


that number by a positive even integer. A root of multiplicity q is here counted 


as q roots. 


Direct application of Descartes’s rule of signs will not isolate the roots of 


an equation, except to give an upper limit to the number of positive and 

negative roots. Descartes’s rule may be used as à rough check on the deriva- 

tion of the general characteristic equation (9.10), since all the roots of this 

equation must be real and positive in factor analysis. This check consists 

simply in noting that, when the characteristic equation is written in order of 

decreasing powers of the variable, the signs of the terms must be alternately 
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plus and minus. The number of variations of signs is then equal to the de- 
gree of the equation, giving the number of positive real roots. Thus, if the 
general equation (F.1) is a characteristic equation, then the coefficients 
must be alternately positive and negative, and the equation may be written 
in the form (9.10). 

For purposes of illustrating the methods of evaluating the roots of a poly- 
nomial equation, the third example of 9.3 will be considered. The charac- 


teristic equation arising in the process of transforming the averoid to the 
principal-factor solution is 


(F.2) f(u) = u* — 11.014243 + 30.5260,? — 31.7349, + 11.1727 = 0. 


In this equation there are four variations of sign, indicating four positive 
real roots. 

More exact methods for isolating the roots of a polynomial equation 
f(u) = 0 will now be presented. One procedure is to plot the polynomial 
f(u), and by means of this graph to isolate the roots. To obtain a reliable 
graph, however, the bend points of the curve are required, and, since these 
points are difficult to obtain for a polynomial of degree greater than three, 
this method is usually impracticable. A much more effective, 
procedure is given by Sturm’s method.* 

The first step in Sturm’s method is to divide f(z) by the first derivative 
f'(u), obtaining a remainder r(x), whose degree is less than that of f’. De- 
noting the quotient by qı, this result may be written f = qf’ + r. Now in- 
dicate the negative of this remainder by f(u), so that f = nf’ — fo. Then 
divide f" by f» and denote the remainder with its sign changed by Jln). 
Continue this process until a constant remainder is obtained. This pro- 
cedure may be exhibited as follows: 


analytical 


f=af' = fay 
T = fe — fis 
(F.3) fe = gfs- fa, 
Jn- = Qn—1fm—1 — fm š 


where fm is a constant. 

If fn = 0 then fm—ı divides f,» and all the other polynomials f;, includ- 
ing f' and f, as may be seen by employing equations (F.3) in reverse order. 
Conversely, any common divisor of f and f' divides f; and hence all the other 
polynomials. Thus if f,, = 0, then fj. is the greatest common divisor of 

* L. E. Dickson, First Course in the Theory of Equations, pp. 75-78. 
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f and f', and a root of fni(u) = 0isa multiple root of f(u) = 0.* On the 
other hand, if fm is a constant different from zero then f and f' have no com- 
mon divisor involving p, and f(u) = 0 has no multiple root. In the remain- 
der of this appendix it will be assumed that f(u) = 0 has no multiple roots, 
so that in the sequence of polynomials, f, f^, fs . - - ı fm, the last one, fm, is 


a constant different from zero. 
Srurm’s THEOREM. If a and b are real numbers, a < b, neither a root of 


f(u) = 0, the number of real roots of f(u) = 0 between a and b is equal to 
the excess of the number of variations of sign of 


(F.4) fla), PG), Fol), «eo fms Sm 


for u = a over the number of variations of sign for p = b. Terms which van- 


ish arc to be dropped out before counting the variations of sign. 
The application of Sturm's Theorem to the isolation of the roots of 
equation (F.2) will now be given. The first derivative of the function in 


(F.2) is 
J'a) = 43 - 33.0426u? + 61.05204 — 31.7349 , 


and the first of equations (F.3) becomes 


f = (25002 — 6884)’ — f2, 


where 
fo = 748356 — 18.2270, + 10.6736 . 


By dividing f by fz, the second of equations (F.3) is obtained, as follows: 


f! = (5345p — 3.1136)f2 — fs, where fs = 1.40464 — 1.4984 . 


Continuing this process, it is found that on the next division the remainder 
is a constant different from zero. The sequence of polynomials (F.4) may 
then be summarized as follows: 


ut — 11.0142,? + 30.52604? — 31.73494 + 11.1727 ; 
4p? — 33.042642? + 61.05204 — 31.7349 , 
(F.5) fo = 7483545 — 18.22704 + 10.6736 , 

fs = 1.40464 — 1.4984 , 

Ja = 2015. 


EX 
pon 
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The roots of f(u) = 0 may then be isolated conveniently as indicated in 
Table F.1. 

In selecting trial values of the variable y, integral values are usually 
taken, beginning with zero, one, and 3o on. According to Sturm's Theorem, 
there is one real root of f(u) = 0 between 0 and 1 since 


Vo-Viz=4—-—3=1, 


where Vu denotes the number of variations of sign of the numbers (F.4) 
when y is a particular real number not a root of f(x) = 0. Trying the next 
integer—2—it is found that the excess of the number of variations of sign 
of the polynomials (F.4) for » = 1 over the number of variations of sign 
for u = 2is two. Consequently, there are two roots between 1 and 2, and 


TABLE F.1 
ISOLATION OF ROOTS or f(u) =0 


s Numper 
TRIAL ee or 
VALUE a COMMENTS 
TIONS 
e £d ge | fz | fs | S| Va 
ah | = | +-+ 4 One root between »=0 and u=1 
—-|-c-l-i-|-4 3 
-|-l4l-4-l4 1 Two roots between 4 —1 and p=? 
Í. +/—-}4+) 4+] 4+ 2 One root between 4 —1.5 and p=2, and 
one root between u—1 and p=1.5 
us wan dnd —|ctic-ic-i-c 1 No roots between »=2 and p=7 
Bread +/+} +] +/+ 0 One root between »=7 and 4-8 


hence these roots are not sufficiently isolated. A value of u halfway be- 
tween these numbers is arbitrarily taken as the next trial value. It is then 
found that V; — Vis = 1 and Vis — Vs = 1 so that one root is isolated 
between 1 and 1.5, and the other between 1.5 and 2. 

At this point, three roots are already isolated. Their total contribution 
to the communality (equal to the sum of the three roots) may be estimated 
to be less than 4. From the averoid solution of Table 8.10 it is known that 
the total contribution of the four principal factors will be about 11. There- 
fore, the remaining factor must contribute upward of 7 to the total com- 
munality, or the remaining root of (F.2) is probably greater than 7. Hence, 
after isolating the three roots in Table F.1, the next trial value of wis taken 
as 7. The last root is isolated between u= 7andy = 8. 

It may be noted that 


(F.6) V.z Vi for ac b. 
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This relation may serve as a rough check on the computation of Sturm’s 
functions (F.4). 

When a particular root of an equation is isolated, it can be computed to 
any number of decimal places by means of Horner’s Method.* This method 
involves successive transformations or reductions of the original equation. 
Thus, to find the root between 7 and 8 of equation (F.2), set u = 7 +d, 
where d is a decimal fraction to be determined. Direct substitution into 


(F.2) gives the transformed equation for d: 
(F.7) d! + 16.9858d + 93.2278d? + 148.5417d — 92.0682 = 0. 


the decimal d, ignore for the moment the 


terms involving d to the second or higher degree. Then from 148.5417d — 
92.0682 = 0, d = 0.6. But this value is too large, since all the terms ig- 
d — 0.5, the polynomial in (F.7) is still positive, 
d = 0.4 + h, where A is of the de- 
f (F.2) may then be writ- 
7.4 +d, where d is 


To obtain an approximation to 


nored are positive. For 
while for d = 0.4 it is negative. Hence 
nomination hundredths. The value of the root o 
ten as a = 7.4 + h, or, to simplify the notation, p = 
now a new decimal of the denomination hundredths. 
The method as just described is laborious especially for equations of high 
degree. If synthetic divisiont is employed instead of direct substitution, 
however, the work can be organized in the simple form indicated in Table 
F.2, In this table, the coefficients and constant term of equation (F.2) are 
recorded in the first row, and the equation is designated by (1). The given 
polynomial is divided synthetically by » — 7, and each quotient is also 
divided by » — 7. Then the suecessive remainders are the coefficients of 
the transformed equation (F.7). These coefficients appear in bold-face type 
just under the first zigzag line in Table F.2, and the fourth-degree equation 
with these coefficients is labeled (2). The polynomial (2), again expressed 
in terms of the variable u, is divided synthetically by 1 — -4, and each quo- 
tient is also divided by » — .4. The remainders are the coefficients of the 
next transformed equation, which is designated by (3) in the table. Again 
consider the variable in the transformed equation as d and ignore the terms 
in d?, d3, and d* in order to get an approximation to the next decimal. Then 
from 231.5331d — 16.6224 = 0, d = .07. The polynomial (3) is positive for 
d = .07 and negative for d = .06. Hence it is divided by .06. This process 
is continued, as completely illustrated in Table F2, until the root of es 
original equation is calculated to as many decimal places as desired. ka 
the present case the root is given correctly to four decimal places by #1 = 


7.4694. 


; 15 
* Ibid., pp. 86-89. + Ibid., pp. 13-15. 
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To clarify further the procedure for evaluating a root of an equation, the 
complete work for the second largest root of (F.2) is presented in Table F.3. 


(1 


— 


(3) 


(4) 


(5) 


TABLE F.2 


CALCULATION OF m BY HORNER'S METHOD 


P nu Constant 


30.5260 — 31.7349 + 11.1727 |7. 


28.0994 + 16.9862 — 103.2409 mW =7+d 


2.4266 — 14.7487] — 92.0682 
20.9006 + 163.2904 | 


+ 

1— 40142 + 
7. T 

+ 


+ 
+ 


+ 


23.3272 | + 148.6417 

852778 92.0682 (2) is — ford = 4 

148.5417 ^ © (2) is + ford =.5 
(2) is + ford = .6 


6.9543 + 40.0728 + 75.4458 [0.4 


93.2278 


1 + 17.3858 + 100.1821 + EU — 16.6224 
4 ar 


m=74+d 
7.1143 + 42.9186 


1+ 17.7858 + 107 
4 "E 


2964| + 231.6331 
743 


1 + 18.1858) + 114.5707 16.6224 _ o7 (3) is — for d = .06 
4 | 231.5331 ` (3) is + for d = .07 
1 + 18.5858 
06 + 1.1187 + 6.9414 + 14.3085 |0.06 
1 + 18.6458 + 115.6804 + 238.4745] — 2.3139 i = 746 + d 
06 — 1.1223 + — 7.0087 
1 + 18.7058 + 110.8117 | + 246.4832 
06 + 1.1259) 
1 + 18.7658| + 117.9376 2.3139 L 009 (4) is — for d = .009 
.06 2454832 — ' (4) is + ford = .01 
1 + 18.8258 
009 + — .1695 + 1.0630 + 2.2189 |0.009 
1 + 18.8348 + 118.1071 + 246.5462) — 0950 in = 7409 + d 
.009 + — .1696 + 1.0045 
1 + 18.8438 + 118.2767| + 247.6107 
.009 + 1697. 
1 + 18.8528| + 118.4464 .0950 L ogogg (8) is — for d = .00038 
:009 247.6107 ` (5) is + for d = .00039 
1 + 18.8618 
0004 + — .0072 + — .0450 +  .0941 |0.00038 
1 + 18.8622 + 118.4536 + 247.6057 — 0009 ui = 7.4694 


One important distinction may be noted between the work in Table F.2 
and that in Table F.3. Whereas the constant terms in the successive trans- 
formed equations of Table F.2 are all negative, those in Table F.3 are all 
positive. In obtaining an approximation to the decimal at each Stage, & 
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different rule must then be applied. In the case of a negative constant 
(Table F.2), the value d is selected which makes the polynomial negative 
just before turning positive. When the constant term is positive (Table 
F.3), however, the value d is selected which just leaves the polynomial 


positive before turning negative. 
TABLE F.3 
CALCULATION OF p: BY HORNER'S METHOD 


ut ra ra x Constant 
(1) 1 — 11.0142 + 30.5260 — 31.7349 + 111727 15 —. 
1.5 — 14.2713 + 24.3821 — 11.0292 ig 15-4 d 
1— 9.5142 4-16.2547 — 7.3528] + 1435 > 
1.5 — 12.0213 + 6.3501 
1— 8.0142 + 4.2334| — 1.0027 
1.5 — 9.7713 . 
1— 6.5142| — 0.5379 4438 4 (2) is + for d = .09 
1.5 10007 ` (2) is — ford =.1 
(2) 1— 6.0142 
09 — 4432 — .5383 — 1387 |0.09 
1— 492412 — 5.9811 — 1.5410| + .0048 m = 1.50 4- d 
09 — 4351 — 5775 
— 4,8842 — 64102| — 2.1186 
09  — 4270 . 
— 6.8432 .0048 _ 002 (3) is + for d = .002 
2118 ` (3) is — for d = .003 
(3) I= 4.6542 
002 —  .0093 — 0137 — _ 0043 0.002 
1-— 4652) — 6.8825 — 2.1922] + -0005 m = 1.592 +d 
002 — 0093 — M37 
1— 4.6502 — 6.8618|— 2-1459 
.002 — -0093 . 
1— 4.6482| — 6.8711 .0005 _ 00023 (4) is ++ ford = 00023 
.002 21459 ` (4)is — ford — .00024 
(4) 1— 4.6462 
0002 — 0011 — -0016 — .0005 0.00023 
.0000 p: = 1.5922 


I- 46460 — 6.8722 — 21475 + 
din Table F.1, can be computed in 


as isolate 
F.3. The values of the four roots 


The two remaining roots, 
a manner similar to that of Tables F.2 or 


May be summarized as follows: 


u = 7.4694, 
= 1.5922, 
(F.8 He 
) p, = 1.0934, 
= 0.8591. 


Bs = 
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The sum of these roots is 11.0141, which agrees with the total contribution 
(11.015) of the four averoid factors of Table 8.10. This is to be expected, 
because the total contribution of the four principal factors must be equal 
to the total contribution of the averoid factors, the former solution being 
merely a rotation of the latter in the same common-factor space. The rela- 


tive size of the contributions of successive factors is different, however, in 
the two cases. 


APPENDIX G 
SOLUTION OF SIMULTANEOUS LINEAR EQUATIONS 


G.1. General Procedure 
The problem of solving a set of simultaneous linear equations arises in 
at least four distinet phases of factor analysis, which will be discussed in 
the following sections of this appendix. In general, a system of n equations 
of the first degree in n unknowns can be solved by means of determinants.* 
While the determinantal method has some undisputed theoretical advan- 
tages, a more economical procedure is desired, especially when dealing with 
a large number of variables. The systems of equations which appear in 
factor analysis have symmetric determinants of coefficients and 80 lend 
themselves to special methods of solution. Gauss's method of substitution] 
produces a routine scheme for the solution of such a set of equations, in- 
cluding a complete check on the arithmetieal work. . 
For convenience of the theoretical work, a system of only three equations 
in three unknowns will be employed. The procedure illustrated, however, 
is applicable to a system of any number of equations, provided the determi- 
nant of coefficients is symmetric. Let the equations be 
aX E bY +ceZ—9 70, 
(G.1) bX 4 dY +eZ—h=0, 
cX t eY 4 fZ - k-0, 


and the remaining letters are constants. 


in which X, Y, Z are the unknowns ! 
Solving the first of these equations for X, 
P PE: g 
(G.2) Beat a 
n e 
and substituting this value into the last two of equations (G.1), they becom 
"XN E: 2-71) 
(a-2)rt ei : 
2 cg 
e); - (x - 2). 
EOAR 
1935, p. 43. 


many textbooks on 
arising in the 


(G.3) 


s Í bra. 
* See, e.g., Maxime Bocher, Introduction t acre dior in 
T This method has been referred to as the md omui equation’, 
statistics, Convenient forms for 2 repa Doolittle and are presented in T. W. 
problem of curve-fitting, were devinen bY servations, 1906, pp. 101-24. 


Wright and J. F, Hayford, The Adjustment of O 
ET 
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Let 


b bc 
d=d-—, a=e-—, fi=f—-t, h-h- 3, mak-F, 


then equations (G.3) may be written in the simpler form 


ciae =h, 


4 
(6 ) aY +fZ = k. 


From the first of these equations, 


-gzh 
(G.5) F=- gta 


and, upon substituting this value in the second equation, it becomes 


ei EPIO 
(G.6) ( T i) Z =i — i. 
Setting 
h 
fr=fi-2 and ky = ky — F> 


the value of Z may be written explicitly as follows: 


(G.7) E 


Then substituting this value of Z into (G.5), the value of Y is determined, 
and, finally, using these values of Y and Z in (G.2), the unknown X is 
determined. 

The substitutions of the last paragraph can be put in a routine outline 
form as indicated in Table G.1. In this form the coefficients and constant 
terms of equations (G.1) are recorded in the appropriate columns of lines 
1, 3, and 7. Since the coefficients of X, Y, and Z in (G.1) constitute a sym- 
metric determinant, and since the method of solution causes terms to the 
left of the principal diagonal to become zero as the variables are eliminated 
these terms are omitted in Table G.1. If there were more equations in the 
system, there would be additional blocks in the table, each successive block 
including one additional line of calculations, 

The directions for the remaining lines of Table G.1 are self-evident 
Specific directions for checking the arithmetic, however, have not been in- 
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cluded. The sum of the entries in each line should be obtained. This value 
should agree, except for rounding errors, with the corresponding expression 


TABLE G.1 
OUTLINE FOR Gauss’s METHOD OF SUBSTITUTION 
Line L, X | Y Z Constant | Check Directions 
Forward Solution 
Du a al $ ce -g zı From (G.1) 
Bucspaceti as vs | x ] - - | g -ls -1 a Zi 
AEE, VE | a@|e jat z; | From (G1) 
2 b P 
ns aan -£ -E M abs; | -himx-m-a 
Deus viis inca od b o em di e —h =: Is+li 
E hy ly; EN 
EES ETVE a -1 E di REM Suy 
a ——————————— 
Wiss min se o [snp dua] sie tine f -=k Zs From (G.1) 
8 Em a |z —£. L; 2 =z a=b 
38 ds ietis dlg ae E FA A A 
9 PE ah Si 2). p ESI exi di 
e rd E 3 j à 
TO} piss v pete xj si ce ort AUR om f: =k: Z; L: + Ls + Ls 
ks E 0m lx 
Y. vx res jumve nei btts ect n d Yr -1 A $e $n 
Back Solution 
ka Entry in “Constant” 
From Ly.....- Z= A column 
2 
e hi Use entries in “Z” and 
From L,...... Y= -72 T “Constant” columns 
1 1 
b Use entries in en 
From L;...... Hee zz +2 on Constant" co 


in the check column. Th 
are merely the sums of the e 
remaining checks (employing 


lements in th 3 
these values) are satis 


e entries in the check colum 


n for lines 1, 8, and 7 
ese lines and are checked if the 
fied. For example, the 
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sum of the elements in line 5 is (di + ei — hı), and this should agree with 
the check sum 


Now, if this check is satisfied, and the sum of the elements in line 4 is 
checked, then the sum Z; must be correct. 

In the back solution of Table G.1 the actual values of X, Y, and Z are 
determined. The value of Z, as given by equation (G.7), is obtained di- 
rectly from the last line of the forward solution. Knowing the value of Z, 
the determination of Y can be effected by means of the values given in line 6 
and agrees with formula (G.5). Finally, employing the values of Y and Z, 
the value of X as given in equation (G.2) can be obtained from the entries 
in line 2 of Table G.1. After Y, Y, Z have been calculated, the back solu- 
tion can be checked by substituting these values into (Gat). 


G.2. Application to Solution of Derived Principal-Factor Pattern 


In transforming the averoid solution of Table 8.10 into a principal-factor 
pattern, systems of equations of the form (9.8) must be solved for the direc- 
tion cosines of the principal factors. To illustrate Gauss's method of sub- 
stitution in this situation, consider the following system of equations for 
the direction cosines of the first principal factor: 


—.0288An — .3136A2 — .2753A4 + .0360Aq = 0, 
—.3136Xn — 6.0744d21 + .0413A3, — 322214 = 0, 
—.2753Mi + .0413A21 — 6.370724 — .0917Au = 0, 

-0360 — 3222M — .0917Aq, — 6.889534 = 0, 


(G.8) 


where the coefficients are taken from (9.22) with the value of u (= 7.4694) 
given by the largest root in (F.8). 

The rank of the matrix of coefficients of (G.8) is three, and the first three 
equations may be used to solve for the first three unknowns, Xi, Aa, Ag, 
in terms of the last unknown, Xa. For all values of the latter, the expres- 
sions for the three unknowns will satisfy the given four equations. Then, 
by employing the auxiliary condition, 


(G.9) Mi + Mi + Mi EM = 1, 


unique values for the four unknowns can be determined. 
The forward solution of the first three of equations (G.8) by Gauss's 
method of substitution is presented in Table G.2. In lines 1, 3, and 7 of 
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this table are recorded the coefficients of the first three of equations (G.8). 
The constant terms are .0360X4, —.3222A4, and —.0917A4, but for sim- 
plicity Aa is omitted in each case and put at the head of the column. The 
directions for calculating the elements in the other lines of this table are 


similar to those given in Table G.1. 


TABLE G.2 


FORWARD SOLUTION FOR DIRECTION COSINES OF 
FIRST PRINCIPAL-FACTOR AXIS 


Line Em An An id Check 
— .2753 .0360 —  .5817 
—9.5590 1.2500 —20.1979 
.0413 — .3222 — 6.3553 
2.9977 — .3920 6.0205 
3.0390 — .7142 —  .8348 
1.1427 — .2085 — .1258 
—6.3707 — .0917 — 6.4624 
2.6316 — .3441 2.2875 
3.4727 — .8161 2.6566 
— .2664 —1.2519 — 1.5183 
-1 —4.6993 — 5.6993 


1 terms of Xa can. be obtained by means 
f Table G.2, ^a = — 4.6993. Then, 


line 6 becomes 


The values of An, a, and Aa i 
of the back solution. From line 11 0 
employing this value, the calculation from 


Ag = 1.142733 — 2685) = — 5.63841 - 


From line 2, 
An = —10.8889A21 — 9.5590^a + 1.250054, 
just calculated, this expression reduces 


dX D 
de irit f X M and ^a into equa- 


and using the values 
ubstituting these values 0 


to Ay = 107.574. 8 
tion (G.9), this condition becomes 


11626M, = 1, 


from which, 
M, = £.0092744 . 
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Using the positive value of A4, all the direction cosines are determined 
uniquely, as follows: 


Au = .9976, 
— M 
(3:10) Xa = —.0436, 
^a = .0093 H 


There are several checks which may be applied. First of all, the sum of 
the squares of the direction cosines (G.10) must be equal to unity, i.e., 


4 
b M, = .9999 . 
s=1 


Then, of course, the \’s must satisfy the equations from which they were 
solved and also the fourth equation of (G.8). Employing the values (G.10) 
in equations (G.8), the left-hand members become .0000, -0000, .0001, and 
— .0027, respectively. These values are considered sufficiently close to zero 
to afford a satisfactory check. The greater discrepancy in the check of the 
fourth equation is due to the fact that the latter was not employed in the 
solution of the \’s. Hence, the values (G.10) may be accepted as the direc- 
tion cosines of the first principal axis with respect to the four averoid axes. 

The direction cosines of the remaining principal axes can be obtained in 
a similar manner by employing the values of ya, p3, and p4 of (F.8) succes- 
sively in (9.22). Three sets of equations of the type (G.8) are thus obtained, 
and from these and auxiliary conditions like (G.9) the values of As2, Asay 
and X, (s = 1, 2, 3, 4) can be determined. The complete sets of direction 
cosines are given in Table G.3. Upon multiplying the averoid pattern of 
Table 8.10 by this matrix of transformation, the derived principal-factor 
pattern of Table 9.1 is obtained. 


TABLE G.3 
MATRIX OF TRANSFORMATION 


| Py Pi 


a] x 
[MN .9977 | —.0543 | .0203 . 0302 
Jie cee) —d0628 — .8274 — .2603 .4944 
i — .0436 —.1895 | .9533 . 2660 
Fes cas cong | .0093 .5413 | —.1500 .8270 


G.3. Application to the Derivation of a Pattern from a Structure 


After a structure, in terms of correlated factors, is obtained, there still 


remains the problem of determining the factor pattern. As indicated in 
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Step 6 of 11.3, the coefficients of the factor pattern may be calculated from 
sets of equations of the form 


ba + bir, s, Pres ih Dim ¥4m aa 


Tiy, , 
bah, cbe iss Bink astm = Ting 5 

(G.11) 
bar, s, + bir, y, Foo e F Dim = Ti 


Boreach j = 1,9... , n) there are m such equations for the determina- 
tion of the coefficients bjt, bio, . . . , Dim Of the m common factors. The terms 
in the right-hand members of these equations are the known structure ele- 
ments. 

The calculation of the b's for any variable z; can be performed by Gauss's 
method of substitution as indicated in Table G.1. Furthermore 
determinant of coefficients of (G.11) is the san 
can be so organized that the solution for all t 
ously. For the illustrative example of 11.3 there are just two factors and 
eight variables, plus two composite variables. The factor coefficients for 
these variables are caleulated in Table G.4. The determinant of coefficients 
of the unknowns (with the elements below the diagonal omitted) appears in 
lines 1 and 3 of Table G.4, while the constant terms of equations (G.11) are 
also recorded in lines 1 and 3 for each variable z;. In the directions these 
elements are designated as correlations with yı Or ys. The directions for 


the calculation of the other elements in the forward solution are similar to 
those given in Table G.1. 


The back solutions for the values of b; and b 
and 2 in the manner indicated in Table G.l, w 
constant terms are not brought over to the sa 
done in (G.1), the negatives of the values in 
6 and 2 must be em 
given by 


, since the 
2e for all values of j, the work 
he b’s can be made simultane- 


j2 are calculated from lines 6 
ith one exception. Since the 
me side of the equation, as is 
the constant columns of lines 
ployed. Thus, for j = 1 the factor coefficients are 


bis = 051 
bu — .484b,. + -919 = .894 . 


ll 


In a similar manner, all the other factor coefficients are calculated. The 
values b5, bj. may be checked upon substitution into equations (G.11). For 
the illustration, the coefficients of the first variable may be checked by sub- 
Stituting b; = .894 and bi; = .051 in the expression 


bu + bisr, s, 


TABLE G4 
CALCULATION OF A PATTERN FROM A STRUCTURE 


CONSTANTS 
Loe DIRECTIONS n y 

a 23 23 n zs ET] 27 zs ui us CuECK 

Forward Solution 
1..... Ty i. „agal .919| .943| .907| .803| .455| .374  .312 .412| .977|  .455| 8.131 
2..... =L —1.| — .484| —.919] —.943| —.907| —.893| —.455| —.374| — -312 —.412| —.977| —.455| —8.131 
3..... Trak Tagal .435| .399| .454| -932| 813). 740). 724 .473| 940) 7.394 
4. — .ABAL, —.445| —.450| —.439  —.432| —.220| —.181| —.151| —.199 —.473| —.220| —3.450 
$5... Lic “939, —.021| —.040) .022| .712| .632  .589  .525 .000  .720 3.94 
6..... Ls/(— .766) |..... =. —.051 mi .052, —.029| —.930 —.825 —.769 —.685|  .000| —.940| —5.150 

-Back Solution 
From line 6 ........ bj .051 — 027 —.082  .029| .930  .825| .769 -685| .000  .940.......- 
From line2 ........ bj .894 956) .932| Am .005| —.025| —.060| .080 .977 SDOO Ius 


7 
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and noting that the result 


894 + .051(.484) = .919 


agrees with ry, . 


G.4. Application to the Estimation of Factors 


Gauss's method of substitution can be adapted to the solution of the re- 
gression coefficients in the estimation of factors. The linear regression of any 
factor F, on the n variables is given by 


(12.6bis) F, = Ban + Baste +... + Binge (8 — 158, a. fi). 
The normal equations for the determination of the 8's are 


Ba + riba +... + inben = be , 
(12.75i*) raba + Baa t+... ore = du, 
Taba + Taba +H... + By = Lus 
where tj, = Tzr, The coefficients of the 8's are the elements of the sym- 
metric matrix of observed correlations. Then the scheme for Gauss's meth- 
od of substitution of Table G.1 can be followed in the present case. 

Since each set of normal equations for the estimation of the successive 
common factors involves the same matrix of coefficients, all the factors can 
be estimated simultaneously. Furthermore, this means that several sets of 
factors, obtained by different methods of analysis, can be predicted at the 
same time. To illustrate, the two principal factors and the two oblique fac- 
tors for the example of eight physical variables are estimated. The com- 
plete work, including checks, is shown in Table G.5. The observed correla- 
tions are taken from Table 7.1; the correlations of the variables with the 
two principal factors P, (General Physical Growth) and P; (Body Type) 
are taken from Table 7.4; and the correlations with the two oblique factors 
yı (Lankiness) and y; (Stockiness) are taken from the factor structure of 
Table 11.2. : z 

After the B's are determined in the “back solution,” the equation of esti- 
mation of any factor may be written. Thus, for the second principal factor, 


the equation of estimation is 


Ps .261z, — .2672; — .2472; — .116z4 + .6062; 
+ .207z + .0882; + .16125, 


TABLE G.5 
ESTIMATION OF FACTORS 


PRINCIPAL OnLiqvE 
VARIABLES Factons Factors 
Live Cueck 
1 2 3 4 5 | 6 | 7 8 BAI] ^ 
Forward Solution 
.859! — 78 -301| .382| .858|— 328] .919| .484| 6.997 
— .839 — .473|— .398|— :301|— “482! 328|— .919|— :484|— 6.997 
[EON NO NOE REUEE EI 
— .727/— .400|- .337|— :255|— 1324 277|— .777|— .409|—5. 
‘099 — :024|- :011|  :022| “002 — 3| 160  020| 540 
— .349|  .085|  .039|— .077|— .324 .482| — .585|— .092| — 2.958 
-801| .380| .319| .237| .345| .810|-.412| .907| .399| 4.786 
— .691 — .381|— .320|— :242|— :308.—.691| :264|— .740|— .390| 4.147 
— .070. .017| .008.— .015|— :065|— 1087] .090|—.117|— .018| — 392 
010 — 016 020|— .028| .032|— 1052| :050|—.009|  :247 
— i190 — :076|- 095|  .133|—.182| :240|— 237| .043|—1.171 
pFENMEEENREE 
-406| — .259|— .328|- . -282|— .789|— .410|-3. 
(008.00 :008/— .032— 043 -048| — -058| — -009|— 125 
.003|— .001|  .004|  .005|—. .010|— .010| :002|— :007 
-035/— .010|  .064|  .010| .039| :001| .036| .031| 425 
.160|  .046|— .292|— :040|— :178|— .005|— .164| — .142| —1.941 
: .162,  .730|  .629| .747| .561| .455| .932| 5.816 
.224|— .188|— .142|— .181|— 406| :155|— .435,— .220| — 1:650 
Wi- m| 39 E E E m) d 
06] 002 — ‘ovel— :002|— :006|— :000,— :008|— 003 — ‘oad 
-767|  .574| -582|  .450| :343| .708| .024| .701| 4.155 
=1. |= .748|- .759|— .595|— .447|— .923| — :031|— :914| —5.417 
x. -583|  .577| .637| .507| .374| .813| 4.491 
.158|— .120|— .152|—:341| .131|— :366|— .193| —1.199 
.004| .005|—.005| :006| .001| 012 
-001|— :001| .000|—:002| :000.— :001 
000| .002| .000| 002] :001|  :008 
— .341|- .257|— .530 — :018| — 524| — 2.534 
.089| .045| .103|—:004| .098| 777 
— .215|- .109|— 249| :010|— 237 — 1.880 
.539| .501| .488| .312| .740| 3.640 
— .115|-:258| .099|— 277|— .146|— .788 
- —.009| .011|—.013|— 002|— :022 
- :003|—.005| .005|—.001|— .003 
= 011|- .000.— .011|—:009|— :053 
m .200|— .537|— .018|— :532| 2:135 
e: .004|— .008| :000|—:008|— .030 
.022| .048.—.002| .042| “609 
- .050|—.109| .005|— :095|— 1:381 
-619| .371| .412| .724| 3.126 
—.328| .125/—:351|—:185|— ‘gas 
—.040. .014|—.054|— :008|— ‘ogg 
:001|—.007| .007|—:001|— :001 
= -902)— -000) — .002|— :001 — .005 
= -204)— .421|— .014|— 417 —1.327 
7.010 —.022| .001|-:021|— :071 
—.003|—.006| .000|—:006|— 023 
en -036/ .084/—.001| :085| “726 
esee —.069|—.161| :002|— 163] 1:391 
Back Solution 
From line 51: Bes=—es (es=element in col. 8)....................]. 10 = 
From line 41: B= — 1328—ee soo eed us supa ane r 
From line 32: 8a = — 0808r. 215a mea L L :091| :207|— 0o09) “Soe pe 
From line 24: Bas = — .7488&— 75986: — .595pas— es... :307| :606| :043| ‘648 ms 
From line 17: B= — .16085-- .04085-- ... — 0468: —e. ` e| .118)/— 1116] :158| :0»4| se 
From line 11: Beas — .19084— .0768n-- ....13384—6..........| ‘116|— 348 :203|— (072 : 
From line 6: Ber — .7048— .3498u-- ... —.3248n—e...........| 1314 ei :391| 2136.2: 
From line 2: Ba = — .846811— .8058ea+ .. . — 3828 —es ; -177|— .261| :275|— 047 A 


o ———M.———————————————— "EN 
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and a particular value of this factor can be obtained by substituting the 
values of the variables for a given individual. The f's, being a solution of a 
system of normal equations of the type (12.7), must satisfy such equations. 
Hence the “back solution" can be checked by substituting the 6’s in equa- 
tions of the type 


(G.12) Bari + Bari +... + Bast is = Tir, U= h Zerga 


where F, is any one of the factors. Employing the first line of Table G.5 
(that is, j = 1), this check for each of the four factors is given by 


v 


858 = rap,, 


8 
Dwar = 484 ns. 


As a measure of the accuracy of estimation of a factor F, by means of 
its regression on the variables, the coefficient of multiple correlation R, may 
be computed. This is given by 


(12,28bis) Re = Bal. + Balos +... + Bantas - 


The multiple correlation can be calculated directly from the values in 
Table G.5, by multiplying the 8's by the first entries in each block for a 
given factor, summing, and taking the square root. Thus, for the first prin- 
cipal factor 


Rp, = V.177(.858) + .314(.849) +... + .069(.619) = V.96277 = .981. 


The multiple correlation coefficients for the other factors can be calculated 
in a similar manner and are given by 


Rp, = .941, R,, = .980, and Ry, = .961. 
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G.b. Application to the Calculation of the Inverse of a Matrix 

In the shortened method of estimation of oblique factors the inverse of 
the matrix of correlations among the common factors is required. The solu- 
tion of this problem can be accomplished by the methods of this appendix. 


For simplicity, suppose there are three common factors and let the matrix 
of their intercorrelations be denoted by 


l ne Tia 
$—]|ra 1 ra 
Ta Tao 1 


From the definition of the inverse of a matrix, it follows that k 


(G.13) oo - I. 


If the elements of the symmetric matrix $7! are denoted b 


Y e’s, equation à 
(G.13) may be written in full as follows: 


l rns n en €; eg 1 0 0 
(G.14) Ta 1 resi |] em ex esj = 10 1 OF , 
ifa ra 1 €31 €32 633 0 0 1 


in which rj, = Tki and ejk = ej. The problem is to determine the elements 
of the inverse matrix from the known correlations, 


l- ent reen + T13€31 
(G.15;) Taen + 1- en + raen 
Taen + raen +1. €31 


1+ ert T1222 F Tiez = 
(G.155) Tali + 1 + ez + T3€32 = 
Ta€m + Taz +1. C32 = 


1. C13 + Tizer + Tiez = 
(G.15;) TnOàs + 1 + e23 + T23€33 = 
ais + Tasos + 1 - ezz 


Moo Qa c oou 
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It will be noted that each set of three equations involves the same matrix 
of coefficients—namely, the correlation matrix $. Hence, the work can be 
80 organized that the solution for all the e's can be made simultaneously. 

To illustrate the procedure for calculating the inverse of a matrix, a 
very simple example will be employed. Consider the following matrix of 
intercorrelations of the two oblique factors, yı and yz, from the analysis of 
the eight physical variables: 


$ = || 1-000 an 
~ | .484 1.000]: 


The inverse of this matrix can be computed by means of Gauss's method 
of substitution. Construct Table G.6, putting the elements of $ (excluding 
those below the diagonal) in lines 1 and 3 of the left-hand portion, and the 
identity matrix in the right-hand portion of the main body of the table. 


TABLE G.6 
CALCULATION OF THE INVERSE OF A MATRIX 


Line Directions d | và Identity Matrix Check 


Tinne zd 3 Ty,k 1 484 1 0 2.484 

2 -Li -1 — .484 -1 0 —2.484 

or an er 1 0 1 2.000 

= MBE, [ec sans — .234 — .484 0 = US 

Is+l, |...... - 766 — .484 1 1,282 

—L;/.766 | cn —1 .632 —1.305 —1.678 

Inverse 

From L; and Le: Calculate ei and é2........., 1.306 Se usas coe tat 
From Ls: Calculate e; and Vitus A98 ris sais omens ey — .682 T0 Pass says site ree 


The calculations are similar to those in the preceding sections of this ap- 
pendix, with the left-hand portion identical to that in Table G.4. The order 
of recording the elements obtained in the “back solution” is reversed from 
that of the preceding examples, however, The calculations from the last 
line of the “forward solution,” although performed first, are put in the last 
line of the “back solution.” This is continued, in reverse order, until the 
calculations from line 2 of the “forward solution” are recorded in the first 
line of the “back solution." In this manner, the final inverse matrix is dis- 
closed in the natural order at the bottom of the table. 


1 


TABLE H.1 
STANDARD ERRORS OF FACTOR COEFFICIENTS 


x V3 -2- so eap 


vg 
p 

N .10 .12 14 -16 18 .20 .22 24 26 .28 .30 .35 -40 AS .50 55 .60 .65 70 75 
20...| .587] .530| .485| .448| .417| .390| .366| .345| .326| .309| .293, .258| .227| .201| .177| .155] .134| .115| .097| .079 
30...| .479| .433| .396| .360| .340| .318| .299| .282| .266| .252) .239| .210| .186| .164| .144| .126| .110} .094| .079| .065 
40...| .415| .375| .343| .317| .295| .276| .259| .244| .231| .218| .207| .182| .161| .142| .125| .109| .095| .081} .068| .056 
50...| .371| .335| .307| .283| .264| .247| .232| .218| .206| .195| .185| .163| .144| .127| .112| .098| .085| .073] .061| .050 
60...| .339| .306| .280| .259| .241| .225| .211| .199| .188| .178| .169| .149| .131| .116| .102| .089| .077| .066| .056| .046 
70...| .314| .283| .259| .239| .223| .208| .196| .184| .174| .165| .157| .138| .122| .107| .094| .083| .072| .061| .052| .042 
80...| .293| .265| .242| .224| .208| .195| .183| .173| .163| .154| .146| .129| .114| .100| .088| .077| .067, .057, .048| .040 
90...| .277| .250| .229| .211| .196| .184| .173| .163| .154| .146| .138| .121] .107| .095| .083| .073, .063| .054| .046| .037 
100...|.262| .237| .217| .200| .186| .174| .164| .154| .146| .138| .131| .115| .102| .090| .079, .069| .060) .051| .043| .035 
g 110.. .| .250| .226| .207| .191] .178| :166| .156| .147| .139| .132| .125| .110| .097| .080| .075| .060, .057| .049| .041| .034 

= 

120.../ .240} .216| .198| .183| .170| .159| .150| .141| .133| .126] .120| .105| .093| .082| .072| .063) .055| .047| .039| .032 
130...| .230| .208| .190| .176| .163| .153| .144| .135| .128| .121| .115| .101| .089| .079| .069| .061| .053| .045| .038  .031 
140...| .222| .200! .183| .169| .158| .147| .138| .130| .123| .117| .111| .097| .086| .076| .067| .058| .051| .043| .036| .030 
150...|.214| .193| .177| .164| .152| .142| .134| .126| .119| .113| .107| .094| .083| .073| .065| .056| .049| .042) .035| .029 
160...| .207| .187| .171| .158| .147| .138| .129, .122| .115| .109| .104| .091| .080, .071| .062| .055| .047| .041| .034| .028 
170...| .201| .182| .166| .154| .143| .134| .126| .118| .112| .106| .100| .088| .078) .069| .061| .053| .046) .039| .033| .027 
180...| .196| .177| .162| .149| .139| .130| .122| .115| .109| .103| .098| .086| .076| .067| .059| .052| .045| .038| .032] .026 
190...| .190) .172| .157| .145| .135| .126| .119, .112| .106| .100| .095| .084| .074| .065| .057| .050| .044| .037| .031| .026 
200...|.186| .168| .153| .142| .132| .123| .116] .109| .103; .098| .093, .081| .072) .064| .056| .049| .042| .036| .031| .025 
250...| .166| .150| .137| .127| .118| .110| .104| .098| .092| .087| .083) .073| .064| .057| .050| .044| .038| .032| .027| .022 
300...| .151| .137| .125| .116| .108| .101| .095| .089| .084| .080| .076| .067| .059| .052| .046| .040| .035| .030, .025| .020 
350...| .140| .127| .116; .107| .100, .093 .088, .083| .078; .074| .070, .062| .054| .048| .042| .037| .032| .027, .023) .019 
400...| .131| .118| .108) .100| .093) .087| .082| .077) .073| .069| .065| .058| .051| .045| .040| .035| .030| .026| .022| .018 
450...| .124| .112| .102| .094| .088| .082, .077| .073| .069| .065| .062; .054| .048| .042| .037| .033| .028| .024| .020| .017 
500...| .117/ .106| .097| .090| .083| .078| .073| .069, .065) .062| .059, .052| .045, .040| .035| .031| .027| .023| .019, .016 
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TABLE H.2 
STANDARD ERRORS OF GENERAL-FACTOR RESIDUALS 


as ic), uod 
o; TEN 5 + 8p + 2p 


p 
MEN <10 | .15 | .20 | .25 | .30 | .35 | .40 | .45 | .50 | .55 | .60 | .65 | .70 |.75 


BU. s scm i - 343) .336).327) .317/.305,.292 .277|.261|.244] .225|.205|.184| .161/.138 
BOs; ausis - 280) .274|.267| .258/.249|.238| .226|.213|.199| .184|.167].150| .132|.112 
A30; ssi à -243| .237|.231| .224|.215,.206| .196/.185|.172| .159.145|.130| .114|.097 
OU s erasa .217,.212/.207| .200|.193..184| .175].165].154] .142..130/.116| .102,.087 
|. - 198. 194).189) . 183|. 176|.168| .160|.151|.141| .130|.118|.106| .093).079 
rA EE -183/.180/. 175| . 169). 163|.156| .148/.139].130| .120|.110!.098| .086).074 
[E -172}.168/.163) .158/.152/.146| .138).130].122} .113].103).092) .081|.069 
Wise sg c - 162).158].154) . 149|. 144.137} .131].123].115] .106/.097|.087| .076).065 
100.......|.154].150]. 146| .142).136].130] .124).117].109} .101|.092/.082] |072)_062 
110.25 35 5s - 146). 143/.139) .135|.130|.124| .118/.111/].104| .096/.087|.078| .069/.059 


.129,.124/.119| .113/.107].099| .092).084!.075| .066/.056 
.124/.120/.114| .109}.102).096} .088/.080.072| .063).054 
.120,.115|.110| .105/.099/.092| .085!.078|.070) .061|.052 
.116/.111/.106| .101/.095/.089| .082.075|.067| .059|.050 
- 112}. 108}. 103} .098,.092/.086| .080 .073).065; .057,.049 


- 109). 105). 100| .095/.090..084| .077|.070!.063) .055).047 
- 106). 102/.097| .092|.087).081) .075).068|.061| .054|.046 
- 103) .099).095] .090).085'.079) .073).067|.060) .052!.045 
- 100) .096).092] .088).083 .077} .071).065].058! .051|.044 
.090/.086/.082| .078).074 .069}. 064/.058/.052| .046|.039 


-082).079).075) .071.067,.063| .058/.053/.047, .042/.036 
-076) .073).070) .066.062..058| .054|.049|.044| .039).033 
-071}.068).065) .062).058 .054| .050).046).041| .036|.031 
d i : .067/.064/.061| .058..055/.051| .047].043|.039| .034/.029 
$00....... . 069) .067/.065) .063/.061|.058| .055). -049| .045,.041|.037 -0321.028 


e 
a 
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TABLE H.3 


SUPPLEMENTARY TABLE FOR COMPUTING 


STANDARD ERRORS OF RESIDUALS 


Pi B-p- Sei +20) | Pi | 2— 2; — $9} + 2p? 
HOB ER 1.467804 ý . 989536 
.04.. ; 1.456128 .33.. .969624 
.05.. i 1.444000 .84.. .949608 
.06.. H LE .95.. .929500 
Eorum " 1.41 .96.. .909312 
.08.. ‘ 1.405024 -37.. . 889056 
*DD.as ui spe 1.391208 .38.. .868744 
«d, us i 1.377000 .89.. .848388 
ecl dus si ux 1.362412 .40.. .828000 
SES necu ne 1.347456 .41. .807592 
m m 1.332144 .42. .787176 
o. 1.316488 .43. .766764 
AO. cave uis 1.300500 .44, . 746368 
SA. usce dem 1.284192 .45.. . 726000 
P i ANE 1.267576 .46.. . 705672 
AB. . i 1.250664 47... - 685396 
A9. . 4 1.233468 248: . .665184 
2D. eus: 1.216000 .49.. 645048 
VEL, geson. 1.198272 -50. - 625000 
«22: vivis 1.180296 sol, - 605052 
BOOS a cia 1.162084 .52. .585216 
ae 1.143648 .58.. .565504 
VO. atio. ns 1.125000 .94.. 545998 
:26.......| 1.106152 .55.. 1526500 
.27.......| 1.087116 <56.. 507232 
.28.......| 1.067904 «Bf. s "488136 
.29.......| 1.048598 258. "469224 
.30.......| 1.029000 ` 
1.009332 .99. - 450508 
: -00933 | -60. .432000 
iM ecc MM 


APPENDIXES 307 
TABLE H.4 
THE PROBABILITY INTEGRAL: AREA UNDER THE NORMAL 
CURVE IN TERMS OF DEVIATES FROM THE MEAN 
1 r/o —AM(r/s)? 
la : 1 
2a Woes f € ar 
ia z ie : ja H ja = ja 

«(20 .0000 .80. 1.60..| .4452 || 2.40..| .4918 | 8.20..| .4993 
02. -0080 || -82. 1.62. .| .4474 || 2.42..| .4922 || 3.22. .1994 

.04. .0160 .84.. 1.64..| .4495 || 2.44..| .4997 3.24..| .49094 
.06. .0239 .86.. 1.66..| .4515 || 2.46..| .4931 || 3.26.. .4994 
.08. .0319 .88.. 1.68..| .4535 || 2.48..| .4934 || 3.28... .4095 
sAQ .0398 .90.. 1.70..| .4554 || 2.50..| .4938 || 3.30.. .4095 
12. .0478 :92... 1.72..| .4578 || 2.52 -4941 || 3.32..| .4995 

M, .0557 .94.. 1.74..| .4591 || 2.54..| .4945 || 3.34..| .4996 
.16....| .0636 1:96... 1.76..| .4608 || 2.56..| .4948 3.36..| .4996 
VAS ess] LOTMA 98... 1.78..| .4625 || 2.58..| .4951 3.38..| .4996 
.20.. -0793 || 1.00.. 1.80..| .4641 || 2.60..| .4953 3.40..| .4997 
.22. .0871 1.02., 1.82..| .4656 || 2.62..| .4956 || 3.42. .4997 
:24....| .0948 || 1.04.. || 1-84..| .4671 || 2.64..| .4959 3.44..| .4997 
BO ss] «1096: |l 1.08... 1.86..| .4686 || 2.66..| .4961 3.46..| .4997 
2 .1108 || 1.08.. 1.88..| .4699 || 2.68..| .4963 | 9.48..| .4997 
.80....) .1179 || 1.10..] .3643 || 1.90..| .4713 || 2.70..| .4965 | 3.50..] .4998 
:82....| .1255 || 1.12..| .3686 | 1.92..| .4726 || 2.72..| .4967 || 3.52..| .4998 
84. -13831 || 1.14..| .3729 || 1.94..| .4738 | 2.74. .4969 || 3.54..| .4998 
:36....] .1406 || 1.16..| .3770 || 1.96.. -4750 || 2.76..| .4971 || 3.56..| .4908 
.38.. - 1480 || 1.18..| .3810 || 1.98.. -4761 || 2.78..| .4973 | 3.58.. .4998 
.40.. -1554 || 1.20..| .3849 || 2.00.. -4772 || 2.80..| .4974 || 3.60.. .4998 
.42.. -1628 || 1.22. .| .3888 i| 2.02..| .4783 || 2.82..| .4976 3.62..| .4099 
A4.. . 1700 1.24 -3925 || 2.04..| .4793 2.84... .4977 || 3.64.. .4099 
.46.. .1772 || 1.26..| .3962 || 2.06.. -4803 || 2.86..| .4979 || 3.66... .4099 
.48.. -1844 || 1.28..| .3997 | 2.08... -4812 || 2.88... .4980 || 3.68.. .4099 
:00.. -1915 || 1.30..! .4032 2.10..! .4821 || 2.90.. -4981 || 3.70..| .4999 
.52.. .1985 || 1.32..| .4066 2.12..| .4830 || 2.92.. .4982 || 3.72..| .4999 
Ti -2054 || 1.34..| .4099 2.14..| .4838 || 2.94.. -4984 || 3.74..| .4999 
-56.. .2123 || 1.36..| .4131 2.16..| .4846 || 2.96.. .4985 || 3.76..| .4999 
.98.. .2190 || 1.38..| .4162 2.18..| .4854 || 2.98.. .4986 || 3.78..| .4999 
.60., -2257 | 1.40..| .4192 || 2.20.. -4861 || 3.00..| .4987 || 3.80..| .4999 
.62.. .2324 || 1.42. 4222 | 2.22..' .4868 || 3.02. . .4987 | 3.82..| .4999 
.64, .2389 || 1.44..| .4951 2.24..| .4875 || 3.04. .4988 || 3.84..| .4999 
.66. -2454 || 1.46..| .4979 2.26..| .4881 || 3.06.. .4989 || 3.86..| .4999 
.68. .2517 1.48..| .4306 2.28..| .4887 || 3.08.. -4990 || 3.88. .4999 
AO... .2580 1.50..| .4332 2.30..| .4893 || 3.10.. -4990 || 3.90. 5000 
2. .2642 1.52..| .4357 2.32..| .4898 || 3.12.. .4991 3.92.. -5000 
S y -2703 1.54..| .4382 2.34. .| .4904 || 3.14.. -4992 || 3.94.. - 5000 
£0, , -2764 1.56..| .4406 2.36..| .4909 3.16..| .4992 3.96..| .5000 
A8. . -2823 |! 1.58..| .4429 2.38..| .4913 || 3.18. -4993 || 3.98..| .5000 
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Adequacy of factor solution 
and choice of communality, 158-59 
and reversal of signs in column of pat- 
tern matrix, 20, 208 
sampling error formulas, 111 
Standard for judging, 22, 151 
Algebraic equations represented by & ma- 
trix equation, 279, 291, 305 
Algebraic solution of symmetric matrix 
completing the square, 94 
diagonal method, 94 
solid staircase method, 94 
Angle of rotation, 216, 220, 239 
Angle of separation of two lines, 57 
Appraisal of factors; see Estimation of fac- 
tors 
Attributes, 10 
Averoid solution 
examples, 194-98 
method of calculation, 194-97 
theoretical development, 192-94 


Axioms for Euclidean geometry, 315 


Barclay, Shepard, 8 
Bartlett, M. S., 283, 285-86 
B-coefficients 
aids in calculation of, 26 
applications of, 31-34 
argument of, 24 
in bi-factor solution, 116, 138, 328 
complementary subset, 24 
definition of, 24 
formula for, 26 
notation, 25 
in oblique solution, 238-39, 249, 261 
use of, 27, 295 
Bi-factor solution 
assumption on residuals, 117 
B-coefficients, 116 
complexity of variables, 106 
examples, 142-53, 169 
final residuals, 121 
form of, 98-101 . 
formula for general-factor coefficients, 
118 
formula for group-factor coefficients, 120 
geometric interpretation, 117 
general-factor residuals, 120-21 
illustration, 118-19 
method of calculation, 328-41 


new pattern plan, 137, 337 
rank of correlation matrix, 116, 137 
reduced pattern, 256 
schematic representation, 100 
stability of, 153-54 
theoretical development, 116-21 
triads in, 120-21, 337-39 
verification and modification, 136-43 
Bipolar factor 
definition of, 104 
in oblique solution, 260 
in principal-factor solution, 104-6, 168, 
296 
Bliss, G. A., 6 : 
Bocher, Maxime, 17, 301, 381 
Burt, Cyril, 104, 156, 167, 176, 180 


Canonical correlation, 5 
Cartesian coordinate system 
coordinate axes, 42 
coordinate hyperplanes, 42 
coordinates, 43 
definition of, 42 
general; see General Cartesian coordi- 
nates 
rectangular; see Rectangular Cartesian 
coordinates 
Center of gravity; see Centroid 
Centroid, 181, 240—41 
Centroid solution 
assumption on residuals, 181 
comparison with principal-factor solu- 
tion, 184 
examples, 189-92 
formula for coefficients of any factor, 
187 
formula for coefficients of first factor, 
182 
formula for coefficients of second factor, 
186 
geometric interpretation, 181 
method of calculation, 353-72 
as preliminary solution, 102 
reduced pattern, 249, 252 
theoretical development, 180-88 
Characteristic equation 
definition of, 161 
in de tnimant form, 160, 204, 206, 209, 
l 
in expanded form, 204, 207, 209, 212 
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* 
Characteristic equation—continued 
roots of, 161, 163-64, 205, 210, 347, 373 
signs of coefficients in, 374 
Coefficient of multiple correlation, 270-71, 
273, 283, 285-86, 312, 391 
Coefficients of pattern identical with ele- 
ments of structure, 18 
Cofactor, 302, 311 
Combinations, notation for number of, 25 
Common factors, 4, 12, 59 
conditions for m, 77-78 
cong itiongy for one, 68-71 
conditions for two, 72-76 
dependence upon set of variables, 107 
number of, 64-68 
theorem on number of, 59 
Common-factor space, 59-63 
determination of; see Chap. IV 
theorem on, 59 
Communality, 13-15, 20, 59, 313 
arbitrary estimates, 157-58 
assumption in calculation of, 115 
complete estimates, 158, 163, 188, 296 
determination of, 78-84 
estimation of, 156-59 
average of all correlations, 157, 192 
highest correlation, 157 
reducing to rank one, 157-58 
triad, 157 
as mean of triads, 115 
proportional to length of vector, 236 
residual, 162 4 


Complete index of factorization, 15 
Complexity of variables 
definition of, 16 = 
in preferred factor solutions, 106 
as a statistical standard, 90 


Components; see Factors 
Composite of matrices, 266 


Composite variables 
correlations, 35-38, 277 
correlations with factors, 241 
correlations with original variables, 38 


definitions of, 34 è 


in estimation of factors, 275-77 
in oblique solution, 241, 246, 249, 255, 
324-25 
standard deviation of, 35, 38, 241 
Composition of variables, 12-15, 288 
Contributions of factors, 13, 160, 163, 211, 
213, 226, 228, 247-48 * 
direct, 247-48 
joint, 247-48 
in relationships among factors, 292 
total; see Total contribution of factor 
Contributions of variable, direct, indirect, 
total, 271, 274 


Coordinate system; see Cartesian coordi- 
nate system 
Coordinates 
definition of, 43 , 
factor coefficients as, 59-60, 181, 216, 
235, 242, 246 
translation of, 46 
Correlation matrix, 19-20, 278 
approximate rank, 157 
cofactors of, 268 
of composite variables, 275 
of factors, 250, 253, 257, 269, 281, 311, 
2 


inverse of, 275, 393 
notation for, 165 
power of, 167, 343-45 
rank of, 68, 76, 78-80, 116, 137, 156, 295 
of variables and factors, 207 
Correlations 
between composite variables; see Com- 
posite variables 
of factors, 237 
linearly independent; see Linear inde- 
pendence of correlations 
reproduced; see Reproduced correlations 
residual; see Residual correlations 
of sums; see Correlations of sums 
Correlations of sums, 34-37 
applications of, 37-38 
see also Composite variables 
Cosine of angle between lines 
and correlation coefficient, 58 
in general coordinates, 55 
in rectangular coordinates, 51-53 
Cramer’s tule; see Determinantal method 
for solution of equations 
Cureton, Edward E., 7-8 


Derived principal-factor solution 
from averoid solution, 212-14 
from bi-factor solution, 209-11 
from centroid solution, 206-8 
examples, 206-14 
labor of computation, 206, 212 
M m of polynomial equation in, 373- 


theoretical development, 202-5 
transformation from initial solution, 


Descartes's rule of signs, 373 
Descriptive variables, 3, 23 
Determinant 
es signs associated with elements, 


of coefficients of set of equations, 204 
cofactor, 302 


columns of, 301-2 
elements of, 301-2 
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expansion of, 77, 303 
minor, 302 x 
of observed correlations, 267-68 
of order n, 302 
principal diagonal, 302, 382 
properties of, 74 * 
rank of, 305 
rows of, 301-2 
of second order, 301 
symmetric, 381-82 
of third order, 301 
Determinantal method for solution o 
tions, 207, 271, 312, 381 
Diagonal matrix 
definition of, 307 
of unique-factor coefficients, 266, 279 
Dickson, L. E., 65, 160, 207, 212, 301, 317, 
374 
Direct factor solution 
assumption on residuals, 87 
calculation of factor coefficients, 86-87 
communalities, 86 
correlation matrix, 
illustration of, 85-88 
pattern plan, 85, 86 
Direct principal-factor sol 
cipal-factor solution 


Direction angles, 49 


f equa- 


ution; see Prin- 


Direction cosines 
conditions on, 218-19, 222 
definition of, 49 
218, 221 


o nce axes, 
of Free iim axes, 239-42, 323-25 
of principal axes, 164, 202-5, 208, 210, 
384-86 
Direction numbers, 51 
Distance, 47 
Distinction between mat 
minants, 304 
Doolittle, M. H., 381 
Doolittle method; see G 
substitution 
Dot product; see Scalar pro 
Doublet, 137, 339 
Dwyer, Paul, 286 


rices and deter- 


auss's method of 


duct 


Efficient factor pattern, 238 

Eight emotional traits 
communalities, 176-77 
intercorrelations, $42. 
principal-factor solution, 

Eight physical variables m 
e oproximative method estimatio! 

oblique factors, 275-76 

B-coefficients, 80, 
bi-factor pattern, 169 


176-77, 342-52 


n of 


^g 
- calculation of communalities, 80-83 
? centroid solution, 192, 239 
complete estimation of oblique 
271-74, 389-91 dép i 
complete estimation of principal fac- 
tors, 389-91 
composi*es of, 241, 275 
estimation of oblique factors by alterna- 
tive principle, 285-86 e 
intercorrelations, 80, 169 5 
means, 272 
multiple-factor solution, 223-260 
oblique factors estimated ar terms of | 
ideal variables, 287-88 — q 
oblique solution, 242, 245 
derivation of pattern from structure, 


386-89 s 
inverse of matrix of correlations of 
factors, 393 
outline for calculation of oblique solu- 
tion, 237-48 
principal-factor solution, 168-73 $ d 
, 


communality, 170 
naming of factors, 17L 


pattern coefficients, 170 
rotation from bi-factor solution, 209- 


11 
rotation from centroid solution, 206-8 
shortened method estimation of oblique 


factors, 280-83 
standard deviations, 272 


values for two girls, 
Eight political variables 
Fommunaliti T d 
description, 17 ^ ^ 
correlation Nm 
li solutio! 
oblique Ffactor solution, 178-79, 200 


E note see Characteristic equa- 


characteristic 4 
aomi see Polynomial equation 
simultaneous linear; see Simultaneous 
linear equations ; 
timation of factors 
Bet yernative principle method, 283-86 
Bartlett's principle, » 
example, 
€ mon factors, 285 —. E 
irinciple of minimizing unique fac 
tors, 289-894 
r$, “ve method, 275-77 r 
approximative me , L0 
tion method, 207-19, 
Jete estimation CIS dual, 268 


for all factor multiple correlation, 


employing pattern matrix, P 
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Estimation of factors—continued 
in terms of observed values, 273 
for uncorrelated factors, 268 
for unique factors, 267-68 
shortened method, 278-83, 392 
assumption on residuals, 278 
assumption on uniquenesses, 278 
coefficient of multiple correlation, 283 
for correlated factors, 279-80 
example, 280-83 
for uncorrelated factors, 280 
. in terms of ideal variables, 286-88 


Euclidean geometry, basic axioms, 41 
Examples; see Illustrative examples 


Factor 
bipolar; see Bipolar factor 
common; see Common factor 
doublet; see Doublet 
estimation of; see Estimation of factors 
general; see General factor 
relationships; see Relationships among 
factors 
specific; see Specific factor 
in standard form, 18 
unique; see Unique factor 
unreliable; see Unreliable factor 
Factor analysis 
applications of, 3-5 
basic data for, 11, 23, 295 " 
geometric interpretation, 59-63, 159, 
314 


indeterminacy of, 23, 289 
nature of, 3 
object of, 289 
order of, 295-96 
and psychologists, 4 
reference system, 289 
and theory, 4, 7 
unity of, 289 
Factor coefficients as coordinates, 59-60, 
181, 216, 235 
Factor pattern 
adequacy of, 21-22 Ww 
coefficients interpreted as coordinates, 
59-60, 181, 216, 235, 242, 246 
definition of, 16 
determined from structure in oblique 
solution, 246-47, 386-88 
distinguished from structure, 234-37 
efficient orthogonal, 238 
notation, 16, 95, 266, 325 
as regression equations, 310-13 
see also Factor structure 
Factor solutions 
averoid; see Averoid solution 
bi-factor; see Bi-factor solution 
centroid; see Centroid solution 
combination of, 103 
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derived principal-factor; see Derived 
principal-factor solution 
direct; see Direct factor solution 
indeterminacy of, 23, 289, 313-15 
invariant, 107-8, 258 
multiple-factor; see Multiple-factor so- 
lution 
oblique; see Oblique solution 
principal-factor; see Principal-factor so- 
lution 
stability of, 153-54, 233, 248, 255, 258 
~ two-factor; see Two-factor solution 
types of, 8 
uni-factor; see Uni-factor solution 
Factor structure 
definition of, 16 
determined by transformation, 244—45 
distinguished from pattern, 234-37 
geometric interpretation of elements of, 
236, 243-44 
matrix form, 237, 266, 326 
see also Factor pattern 
Factors of mind, 4-5 
Filon, L. N. G., 123 
Fisher, R. A., 135 
Five physical variables 
B-coefficients, 140 
correlations, 139 
doublet, 142 
final bi-factor pattern, 142 
general-factor coefficients, 139 
general-factor residuals, 139 
modification of pattern plan, 140 
new pattern plan, 141 
standard error of general-factor coef- 
ficients, 140-41 
standard errors of residuals, 141 
Flat spaces, 42, 46° 


Gauss’s method of substitution, 271 
in calculation of inverse matrix, 392-93 


in derivation of pattern from structure 
386-88 


in Pe tred principal-factor Solution, 384— 
9 
in estimation of factors, 282, 284, 389-91 
outline for, 383 
theoretical development, 381-84 
General Cartesian coordinates 
elementary formulas in, 53-55 
in oblique solution, 236 
General factor, 4, 12, 107 
in bi-factor solution, 99, 106 
in principal-factor solution, 104, 106 
General rigid motion, 49 
Geometric fit 
ellipsoidal, 94, 104 
hyperplanar, 92-93, 102 
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linear, 91-92, 05, 90 
in multiple-factor solution, 223 
planar, 92, 99 

Geometrie interpretation of 
bi-factor plan, 99, 117 
correlation, 
dimensionality, 41 
elements of factor structure, 236-37 
factor analy 59-03, 159, 314 
indeterminacy of factor solutions, 314 
oblique solution, 234-37, 239 
pattern vs. structur j 
statistical variables, 57, 92, 216 . 
two solutions of eight physical vari- 

ables, 171-73 

Geometry of higher dimensions, 41-13 

Girshick, M. A., 323 

Gosnell, Harold F., 177 

Group factor, 12 
in bi-factor solution, 101, 106 : 
in multiple-factor solution, 102, 106 
in uni-factor solution, 96-97, 106 

Groups of variables, 28, 96, 101, 295, 328- 


Harman, Harry H., 18, 19, 34, 275, 278, 
289 

Hayford, J. F., 381 

Holzinger, Karl J., 18, 19, 34, 94, 101, 112, 
144, 274, 289, 309, 311-12 

Homogeneous coordinate system, 42 

Homogeneous linear equations of rank 
(n— D), 161 

Horner's method, 377-79 

Hotelling, Harold, 5, 104, 155, 156, 159, 
163, 105, 210, 313 

Hyperplane, 41-43 

Hypothetical design, 23 


Ideal variables, 205, 286-87 
Identity matrix, 280, 293, 307, 392-93 
Illustrative examples: see under each of 
the following: ; 
Eight emotional traits 
Eight physical variables 
Eight olitical variables 
Five physical variables 
Thirteen psychological variables 
Twenty-four psychological variables 


Index of completeness of factorization, 15 
for twenty-four psychological variables, 
150 
Inner product; see Scalar product 
Intercorrelations of variables, 3 


Invariant lines (principal axes), 164 

Inverse matri: 278-81, 287, 290 
cofactors in, 30 
computation of. 
definition of, 30: 
of product of matrices, 308 


Jackson, Dunham, 133 


, 203, 308, 392-93 


Kelley, Truman L., 3, 4, 22, 72, 94, 155 
Kendall, M. G., 93, 315 
Kronecker delta, 49, 219, 314 


Lagrange's multipliers, 203, 322 

Lamb, Horace, 108 

Laplace's development of a determinant, 
212 


Law of cosines, 52, 54 
Ledermann, Walter, 65, 72, 278 
Linear combination and dependence, 43- 
16 
Linear dependence 
definition of, 43-44 
theorem on, 45, 316-17 
transitive law for, 46 
Linear equations; see Simultaneous linear 
equations 
Linear independence 
of columns of matrix, 318 
of correlations, 64-66 
definition of, 44 
of equations, 210 
of rows of matrix, 319 
Linear spaces, 42, 45 
theorem on, 46 


MeMahon, James, 94 
Mathematical theory and observed data, 
xd 
Matrix 
cofactors of, 268 
column vector, 266, 286, 290, 319 
correlation; see Correlation matrix 
definition of, 303 
definition of rank of, 305 
determinants of, 305-6, 317 
MU of, 306 
elements as rectan i 3 
veces Eular Cartesian co. 
equation, 279, 287, 291, 305 
uL, A 293, 307, 392-93 
inverse of, 275, 278-81, 287, $ -9 
ae ae 87, 290, 203-04, 
multiplication of, 205, 343-45 
column-by-column, 280-81 
commutative law, 306 
row-by-column, 280, 204, 306 
row-by-row, 294 
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Matrix—continued 
nonsingular, 305, 308, 319 
notation, 19, 165-67, 217-18, 220-21, 
265, 303-4 
one-rowed, 304 
order of, 278, 287 
orthogonal, 49, 217-18, 222 
pattern; see Pattern matrix 
postmultiplication of, 227, 279, 294, 
306-7, 325 
premultiplication of, 278-79, 287, 290- 
91, 306-7 
principal diagonal, 280 
product by scalar, 307 
product by transpose, 58, 317-19 
rank of symmetric, 65 
of reproduced correlations, 19, 278 
diagonal elements, 20, 278 
residual, 166-67, 186-87 
scalar, 307 
singular, 305 
square, 303, 305 
sum of, 282, 306 
symmetric, 280, 287, 305, 328, 389, 392 
theorems involving rank of, 45, 46, 58, 
316-19 
of transformation, 205, 208, 210, 217, 
220-21, 244-45, 290, 294, 386 
transpose of, 19, 58, 280, 285, 291, 304, 
305, 308, 319 
triangular, 247 
Maximal linearly independent set, 318 
Mean value of certain ratios, 124-25, 320- 
21 
Miller, W. L., 198 
Minor, 302, 311, 318 
Moulton, F. R., 23 
Mullen, Frances, 80, 138, 168, 169 
Multiple-factor solution 
complexity of variables, 106, 223, 229 
examples, 223-33 
form of, 101-3 
geometric fit, 223-24, 229-30, 232 
rotation of preliminary solution, 102, 
180, 222-23 
stability of, 233 


Norm, 47 


Normal equations, 267, 270-71, 276, 389, 
391 


Normal law, equation of, 132, 397 
Normally distributed factors, 313, 315 
Notation 

B-coefficients; see B-coefficients, nota- 

tion 

coordinates of points, 43 

elements in different spaces, 59-63 

in Horner’s method, 377 


matrix; see Matrix notation 

number of combinations, 25 

in principal-factor method, 348 
reflection of variables, 185 

for regression coefficients, 267 
set-theory; see Set-theory notations 
summational; see Summational notation 


N-tuple; see Point 


Oblique reference system, 235, 238 
Oblique reference vectors 
angle between, 236, 243 
correlation between, 237 
direction cosines of, 239-42 
variances of, 242 
Oblique solution 
angle between reference axes, 236 


comparison for different initial patterns, 
249, 258 

examples 
based upon bi-factor patterns, 255-59 
based upon centroid patterns, 249-55 
of inapplicability, 260-61 

geometric basis, 234-37 

outline for calculation, 237-48 
contributions of factors, 247-48 
direction cosines of axes, 239-42, 323- 

25 


factor pattern, 246-47 
factor structure, 243-46 
initial orthogonal pattern, 238 
M — ues of factors, 242-43, 
6 
subsets of variables, 238-39 
parts of, 237 


pattern derived from structure, 246-47, 
386-88 


rotation of orthogonal solution, 234, 237 

stability of, 248, 255, 258 
Order of rotations, 219, 222 
Orthogonal matrix, 49, 217-18, 222 
Orthogonal reference vectors 

in bi-factor solution, 117 

in centroid solution, 181 

in multiple-factor solution, 222 

in principal-factor solution, 202 
Orthogonal rotations; see Orthogonal 

transformations 

Orthogonal solution, 90-91, 181, 202 
Orthogonal system, 202, 220, 232 

total contribution of, 233 
Orthogonal transformations 

definition of, 49 

in higher-dimensional space, 221-22 

in matrix notation, 217 

notation in, 216-26 

order of, 219 

in plane, 215-18 

to principal-factor axes, 202-14 
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product of, 219 k 
theoretical development, 215-22 
in three-space, 218-21 

Osgood, William F., 202, 322 


Parametrie equations of line, 50, 51 
Pattern; see Factor pattern 
Pattern matrix, 19-20, 208, 326 
centroid, 206, 208, 225 - 
common-factor portion, 266, 281, 286 
complete, 266 
definition of, 19 
in estimation of factors, 269 
principal, 205, 208, 211 
of uniquenesses, 281 
Pearson, Karl, 123 
Pencil of planes, 99, 117 
Pentad criterion, 72 
Poincaré, H., 23 
Point 
arithmetic, 42 
geometric, 42 
Point representation of variables, 57, 93- 
94, 159 
Polynomial equation 
isolation of roots, 373-77 
sequence of, 375 
solution of, 373-80 
Postmultiplication, 227, 279, 294, 325-26 
Preferred forms of solution, 8, 265, 289, 
296 
assumptions and properties of, 106-8 
standards for judging, 89-94 
composition of variables, 90 
contributions of factors, 91 
geometric fit, 91-94 
parsimony, 90 
uncorrelated factors, 90-91 
Preferred system of reference, 89 
Premultiplication, 278-79, 287, 290-91 
Primary abilities, 102, 107 
Primary factors; see Multiple-factor solu- 
tion 
Principal components; see Principal-factor 
solution 
Principal-factor solution 
assumption on residuals, 160 
complexity of variables, 106 
derivation of method, 322-23 
derived; see Derived principal-factor 
solution 
examples, 168-79 
form of, 103-5 
iterative method of calculation, 163- 
68, 342-52 
labor of computation, 174-75, 179, 206, 
212 


orthogonality of coefficients, 163 

theoretical development, 159-63 
Probability integral, 132, 397 
Product of matrices 

inverse of, 308 

rank of, 58 

transpose of, 308 
Product matrix, 221 
Product of rotations, 219, 221 
Projections of lines, 216-17 
Projective axiom, 315 
“Pythagorean” relation, 15 


Radius vector, 54 

Reciprocal of matrix; see Inverse matrix 

Rectangular Cartesian coordinates 
elementary formulas in, 47-53 

Reduced pattern, 242, 245, 255 

Reduced structure, 245, 276 


Reference axes; see Reference vectors 
Reference vectors 
centroid, 181 
direction cosines of, 164, 202, 208 
and linear fit, 92 
oblique; see Oblique reference vectors 


orthogonal; see Orthogonal reference 
vectors 


principal, 202 
unit, 235, 289 
Reflection of axis, 226, 227 
Reflection in the origin; see Reflection of 
variables 
Reflection of variables, 105, 184-88, 295, 
361-66 
Regression coefficients, 17, 267, 311 
Regression equations, 17, 265, 276- 
311213 gom 
Rela 3 between pattern and struc- 
ure ' 
geometric distinction, 234-37 
matrix formulation, 17, 266, 269, 325-27 
Relationships among factors, 289-92 
example, 292-95 
matrix formulation, 291 
transformation of coordinates, 292, 294 
used in estimating factors, 292 
Relative contributions of factors, 91 
Reliability, 14-15, 295 
Reproduced correlations, 19, 21 
from bi-factor solution, 101 
matrix of, 19, 278 
related to observed correlations, 21, 60, 
265 
as scalar products, 61 
from two-factor solution, 112 
from uni-factor solution, 97 
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Residual correlations 
definition of, 21 
final, 121, 136 
first-factor, 162 
general-factor, 120, 135 
general notation, 162, 187 
matrix of, 166-67, 186-87, 335 
standard error of, 128-32 

Residual-factor space, 120, 183, 187 

Residual vectors, 184 

Residuals; see Residual correlations 

Rotation; see Transformation 


Sampling errors 
approximations to, 129-31, 391-96 
of communality, 123-24 
of factor coefficient, 122-32, 147-48, 
226, 394 
interpretation of, 132-36 
in judging factor solutions, 111 
of residual, 128-32, 149, 395-96 
of triad, 122-23 
Scalar, 325 
definition of, 307 
product by matrix, 307 
Scalar matrix, 307 
Scalar product of two vectors, 47, 53, 181 
183, 202 
Scatter diagram, 57 
Schmidt, Margaret, 177 
Scientifie parsimony, 3 
Scientific theory, parts in, 6 
Sedecimant, definition of, 252 
Set-theory notations 
definition of, 96 
illustration of, 97 
use of, 113, 115, 118-21, 321, 328-30 
Simple structure, conditions for, 102 
Simultaneous linear equations 
solution by determinants, 267, 271. 312, 


381 
Solution by Gauss's method of substitu- 
tion, 381-93 


Sisam, C. H., 219 
Snyder, Virgil, 219 
Sommerville, D. M. M. 41 
Spearman, Charles, 4, 34, 68, 98, 101, 107, 
111-12, 167, 296, 309 
Specific factors, 14-15 
Specificity, 14-15 
Standard error of function in terms of 
standard errors of variables, 319-20 
Standard error of a variance, 82 
Standard errors; see Sampling errors 
Statistical variables 
design of, 23, 83, 261, 295 
interpreted as vectors, 40, 57 


as linear functions of factors, 12, 16, 59, 
60, 265 
observed values, 10, 265 
point representation, 57 
in standard form, 11, 13 
standardized values, 11, 273 
vector representation, 57 
as vectors projected into common-factor 
space, 60, 235 
Structure; see Factor structure 
Sturm's method, 374 
Sturm's theorem, 375-76 
Summational notation, 10, 25, 54, 96, 113, 
353 
Swineford, Frances, 101, 144, 274, 309, 328 
System of elements, 96, 117 
System of numbers; see Point 


Tests of significance, 132-36 
Tetrads, 4, 68, 112 
Thirteen psychological variables 
averoid solution, 194-97 
bi-factor solution, 153-54 
centroid solution, 189-90, 353-72 
correlations, 30 
multiple-factor solution, 226-29 
intermediate coordinates, 227 
naming of factors, 229 
total contribution of factor system, 
228 
oblique solution based on bi-factor pat- 
tern 
contributions of factors, 258 
correlations of factors, 257 
direction cosines of axes, 256 
factor pattern, 257 
factor structure, 257 


oblique solution based on centroid pat- 
tern 


angles between axes, 250 
contributions of factors, 251 
correlations of factors, 250 
direction cosines of axes, 249-50 
factor pattern, 251 
factor structure, 251 
reduced bi-factor pattern, 256 
reduced centroid pattern, 249 
relationships between  bi-factor and 
oblique solutions, 292-95 
Thomson, G. H., 283 


Thurstone, L. L., 5, 19, 22, 34, 59, 94, 102, 
rus 150, 158, 180, 184-85, 188, 215, 


Total contribution of factor, 13, 160, 162 
228, 248, 322, 352, 376,'380 i 
Transformation 
linear, 48 
matrix, 205, 208, 210 
in oblique solution, 243 
orthogonal, 49, 202-5, 215-22 


dibus 
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in principal-factor solution, 163-65, 
201-5 
of scale, 273-74 
theorem on, 49 
Transpose of matrix, 280, 285, 291, 305 
definition of, 19, 304 
notation for, 304 
theorems on, 58, 308 
Triads - 
in bi-faetor solution, 120-21, 337-39 
definition of, 71 = 
in estimation of communality, 115, 157 
standard error of, 122-23 
in two-factor solution, 112, 114-15 
Tucker, Ledyard, 19 
Twenty-four psychological variables. 
approximative method of estimation of 
bi-factors, 276-77 
averoid solution, 198 
B-coeffivients, 32 
bi-factor solution, 143-53, 328-41 
communality, 150 : 
comparison of factor coefficients, 146 
doublet, 149 
naming of factors, 151-52 - 
new pattern plan for six variables, 145 
pattern, 144 
specificity, 150 
standard errors of coefficients, 147-48 
standard errors of residuals, 149 
uniqueness, 150 
unreliability, 150 
centroid solution, 190-91 
composite variables, 37-39 
correlations, 30 
derived principal-factor solution, 212-14 
direction cosines of principal axes, 384— 
86 
first two principal factors obtained di- 
rectly, 174-75 
groups of variables, 33 
means, 29 
multiple-factor solution, 229-33 
intermediate coordinates, 231 
level distribution of variance, 231 
naming of factors, 232-33 
orthogonal systems, 232 
total contributions of factor systems, 
233 
oblique solution based on bi-factor pat- 
tern 
contributions of factors, 259 
correlations of factors, 257 
direction cosines of axes, 256 
factor pattern, 259 
factor structure, 259 
oblique solution based on centroid pat- 
tern 
contributions of factors, 254 
correlations of factors, 253 


direction cosines of axes, 253 
factor pattern, 
factor structure 
reduced uni-factor pattern, 255 
reduced bi-factor pattern, 256 
reduced centroid pattern, 252 
reliability coefficients, 29 
roots of characteristic equation, 374-80 
standard deviations, 29 


Two-factor solution, 95, 101, 111-15 
method of triads, 114-15 
Spearman’s method, 112-14 

Two-factor theory, 4, 112 


Uncorrelated factors, 13 
Uni-factor solution 

complexity of variables, 106 

form of, 95-98 

ideal pattern, 97 

reduced, 255 

schematic representation, 96 
Unique factor, 4, 12, 266, 283 
Uniqueness, 13-14, 313 

in shortened method of estimation of 

factors, 278 

Unit reference vectors, 235, 289 
Unitary Traits Committee, 94 
Unreliable factors, 14-15 


Variables 
composite; see Composite variables 
descriptive; see Descriptive variables 
statistical; see Statistical variables 
Variation of sign, 373-76 
Vector representation of variables, 57, 91- 
Vectors 
addition of, 43 
column, 266, 286, 290, 319 
cosine of angle of inclination, 58-61 
linear combination of, 43 
multiplication by numbers, 43 
projected into common-factor space, 60 
reference; see Reference vectors 
unit, 235, 289 
zero, 319 


Wenger, M. A., 309 
Wien in stop factoring, 151, 158, 188, 194, 


see also Adequacy of factor solution 
Whitehead, J. H. C., 48 
Wilcox, L. R., 96 
Wright, T. W., 381 
Yule, G. Udny, 93, 315 
Zero vector, 319 
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